osdfs
Subsystems (4)
cover osdfs
Please review an OSD based file system.
Given that our OSD initiator library is accepted into Kernel, we would like to also submit an osdfs. This is the first iteration of this file system.
The next stage is to make it exportable by the pNFS-over-objects Server. osdfs is one of the building blocks for a full, end-to-end open source reference implementation of a Server/Client pNFS-over-objects we want to have available in Linux. Other parts are the Generic pNFS client project with the objects-layout-driver, and the generic pNFS server plus osdfs once it is adapted to be exportable.
(See all about pNFS in Linux at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_prototype_design)
osdfs was originally developed by Avishay Traeger <[email protected]> from IBM. A very old version of it is hosted on sourceforge as the osdfs project. It was originally developed for the 2.6.10 Kernel over the old IBM's osd-initiator Linux driver.
Since then it was picked by us, open-osd, and was both forward ported to current Kernel, as well as converted to run over our osd Kernel Library. The conversion effort, if anyone is interested, is also available as a patchset here:
git-clone git://git-open-osd.org/open-osd.git osdfs-devel
or on the web at:
http://git.open-osd.org/gitweb.cgi?p=open-osd.git;a=shortlog;h=refs/heads/osdfs-devel
The Original code is based on ext2 code from the Kernel at the time. Further reading is available at the last patch in the osdfs.txt file.
I have mechanically divided the code in parts, each introducing a group of vfs function vectors, all tied at the end into a full filesystem. Each patch can be compiled but it will only run at the very end. This was done for the hope of easier reviewing.
Here is the list of patches
[RFC 1/9] osdfs: osd Swiss army knife
[RFC 2/9] osdfs: file and file_inode operations
[RFC 3/9] osdfs: symlink_inode and fast_symlink_inode operations [RFC 4/9] osdfs: address_space_operations
[RFC 5/9] osdfs: dir_inode and directory operations [RFC 6/9] osdfs: super_operations and file_system_type [RFC 7/9] osdfs: mkosdfs
[RFC 8/9] osdfs: Documentation
Patches to be submitted
[PATCH 9/9] [out-of-tree] open-osd: Global Makefile and do-osdfs test script
This patch will not be submitted. It is only needed if compiling out-of-tree. One more patch that is missing from this patchset is the patch to Kernel's fs/Makefile fs/Kconfig and Documentation/filesystems/00-INDEX.
This patchset is also available on:
git-clone git://git-open-osd.org/open-osd.git osdfs
or on the web at:
http://git.open-osd.org/gitweb.cgi?p=open-osd.git;a=shortlog;h=refs/heads/osdfs
If anyone wants to actually run this code and test it then please start here: http://open-osd.org
and also the osdfs.txt file at last patch should help
Thanks in advance
Boaz
RFC 1/9 osdfs: osd Swiss army knife
In this patch are all the osd infrastructure that will be used later by the file system.
Also the declarations of constants, on disk structures, and prototypes.
And the Kbuild+Kconfig files needed to build the osdfs module.
--- fs/osdfs/Kbuild | 24 ++++ fs/osdfs/Kconfig | 13 ++ fs/osdfs/common.h | 154 ++++++++++++++++++++++++++ fs/osdfs/osd.c | 316 +++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/osdfs/osdfs.h | 180 ++++++++++++++++++++++++++++++ 5 files changed, 687 insertions(+), 0 deletions(-) create mode 100644 fs/osdfs/Kbuild create mode 100644 fs/osdfs/Kconfig create mode 100644 fs/osdfs/common.h create mode 100644 fs/osdfs/osd.c create mode 100644 fs/osdfs/osdfs.h
fs/osdfs/Kbuild new file +24 -0
@@ -0,0 +1,24 @@ +# +# Kbuild for the OSDFS module +# +# Copyright (C) 2008 Panasas Inc. All rights reserved. +# +# Authors: +# Boaz Harrosh <[email protected]> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 +# +# Kbuild - Gets included from the Kernels Makefile and build system +# + +ifneq ($(OSD_INC),) +# we are built out-of-tree Kconfigure everything as on +CONFIG_OSDFS_FS=m +EXTRA_CFLAGS += -DCONFIG_OSDFS_FS -DCONFIG_OSDFS_FS_MODULE +EXTRA_CFLAGS += -I$(OSD_INC) +# EXTRA_CFLAGS += -DCONFIG_OSDFS_DEBUG +endif + +osdfs-objs := osd.o +obj-$(CONFIG_OSDFS_FS) += osdfs.o
fs/osdfs/Kconfig new file +13 -0
@@ -0,0 +1,13 @@ +config OSDFS_FS + tristate "OSD based file system support" + depends on SCSI_OSD_ULD + help + OSDFS is a file system that uses an OSD storage device, + as its backing storage. + +# Debugging-related stuff +config OSDFS_DEBUG + bool "Enable debugging" + depends on OSDFS_FS + help + This option enables OSDFS debug prints.
fs/osdfs/common.h new file +154 -0
154 lines not on this page · raw message
fs/osdfs/osd.c new file +316 -0
316 lines not on this page · raw message
fs/osdfs/osdfs.h new file +180 -0
180 lines not on this page · raw message
Signature · 1 line
1.6.0.1
RFC 2/9 osdfs: file and file_inode operations
In reply to Boaz Harroshimplementation of the file_operations and inode_operations for regular data files.
All file_operations are generic vfs implementations except osdfs_truncate.
--- fs/osdfs/Kbuild | 2 +- fs/osdfs/file.c | 58 ++++++++++++++++++++++ fs/osdfs/inode.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/osdfs/osdfs.h | 11 ++++ 4 files changed, 210 insertions(+), 1 deletions(-) create mode 100644 fs/osdfs/file.c create mode 100644 fs/osdfs/inode.c
fs/osdfs/Kbuild +1 -1
@@ -20,5 +20,5 @@ EXTRA_CFLAGS += -I$(OSD_INC) # EXTRA_CFLAGS += -DCONFIG_OSDFS_DEBUG endif -osdfs-objs := osd.o +osdfs-objs := osd.o inode.o file.o obj-$(CONFIG_OSDFS_FS) += osdfs.o
fs/osdfs/file.c new file +58 -0
58 lines not on this page · raw message
fs/osdfs/inode.c new file +140 -0
140 lines not on this page · raw message
fs/osdfs/osdfs.h +11 -0
@@ -177,4 +177,15 @@ int extract_list_from_req(struct osd_request *req,void free_osd_req (struct osd_request *req); +/* inode.c */ +void osdfs_truncate (struct inode *inode); +int osdfs_setattr (struct dentry *,struct iattr *); + +/********************* +* operation vectors * +*********************/ +/* file.c */ +extern struct inode_operations osdfs_file_inode_operations; +extern struct file_operations osdfs_file_operations; +#endif
Signature · 1 line
1.6.0.1
RFC 3/9 osdfs: symlink_inode and fast_symlink_inode operations
In reply to Boaz HarroshGeneric implementation of symlink ops.
--- fs/osdfs/Kbuild | 2 +- fs/osdfs/osdfs.h | 4 +++ fs/osdfs/symlink.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 59 insertions(+), 1 deletions(-) create mode 100644 fs/osdfs/symlink.c
fs/osdfs/Kbuild +1 -1
@@ -20,5 +20,5 @@ EXTRA_CFLAGS += -I$(OSD_INC) # EXTRA_CFLAGS += -DCONFIG_OSDFS_DEBUG endif -osdfs-objs := osd.o inode.o file.o +osdfs-objs := osd.o inode.o file.o symlink.o obj-$(CONFIG_OSDFS_FS) += osdfs.o
fs/osdfs/osdfs.h +4 -0
@@ -188,4 +188,8 @@ int osdfs_setattr(struct dentry *, struct iattr *);extern struct inode_operations osdfs_file_inode_operations;extern struct file_operations osdfs_file_operations; +/* symlink.c */ +extern struct inode_operations osdfs_symlink_inode_operations; +extern struct inode_operations osdfs_fast_symlink_inode_operations; +#endif
fs/osdfs/symlink.c new file +54 -0
54 lines not on this page · raw message
Signature · 1 line
1.6.0.1
RFC 4/9 osdfs: address_space_operations
In reply to Boaz HarroshOK Now we start to read and write from osd-objects, page-by-page. The page index is the object's offset.
--- fs/osdfs/inode.c | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/osdfs/osdfs.h | 3 + 2 files changed, 287 insertions(+), 0 deletions(-)
fs/osdfs/inode.c +284 -0
@@ -60,6 +60,290 @@ int osdfs_get_block(struct inode *inode, sector_t iblock,return 0 ; } +/* +* Callback function when writepage finishes. Check for errors, unlock, clean +* up, etc. +*/ +void writepage_done (struct osd_request *req,void *p) +{ +int ret; +struct page *page = (struct page *)p; +struct inode *inode = page->mapping->host; +struct osdfs_sb_info *sbi = inode->i_sb->s_fs_info; + + ret =check_ok (req); +free_osd_req (req); +atomic_dec (&sbi->s_curr_pending); + +if (ret) { +if (ret == -ENOSPC ) +set_bit (AS_ENOSPC , &page->mapping->flags); +else +set_bit (AS_EIO , &page->mapping->flags); + +SetPageError (page); + } + +end_page_writeback (page); +unlock_page (page); +} + +/* +* Write a page to disk. page->index gives us the page number. The page is +* locked before this function is called. We write asynchronously and then the +* callback function (writepage_done) is called. We signify that the operation +* has completed by unlocking the page and calling end_page_writeback(). +*/ +static int osdfs_writepage (struct page *page,struct writeback_control *wbc) +{ +struct inode *inode = page->mapping->host; +struct osdfs_i_info *oi =OSDFS_I (inode); +loff_t i_size =i_size_read (inode); +unsigned long end_index = i_size >>PAGE_CACHE_SHIFT ; +unsigned offset =0 ; +struct osd_request *req =NULL ; +struct osdfs_sb_info *sbi; +uint64_t start; +uint64_t len =PAGE_CACHE_SIZE ; +unsigned char *kaddr; +int ret =0 ; + +if (!PageLocked (page)) +BUG (); + +/* if the object has not been created, and we are not in sync mode, +* just return. otherwise, wait. */ +if (!ObjCreated (oi)) { +if (!Obj2BCreated (oi)) +BUG (); + +if (wbc->sync_mode ==WB_SYNC_NONE ) { +redirty_page_for_writepage (wbc, page); +unlock_page (page); + ret =0 ; + goto out; + }else { +wait_event (oi->i_wq,ObjCreated (oi)); + } + } + +/* in this case, the page is within the limits of the file */ +if (page->index < end_index) + goto do_it; + + offset = i_size & (PAGE_CACHE_SIZE -1 ); + len = offset; + +/*in this case, the page is outside the limits (truncate in progress)*/ +if (page->index >= end_index +1 || !offset) { +unlock_page (page); + goto out; + } + +/* otherwise, the page straddles i_size. It must be zeroed out. */ + kaddr =kmap_atomic (page,KM_USER0 ); +memset (kaddr + offset,0 ,PAGE_CACHE_SIZE - offset); +flush_dcache_page (page); +kunmap_atomic (page,KM_USER0 ); + +do_it: +BUG_ON (PageWriteback (page)); +set_page_writeback (page); + start = page->index <<PAGE_CACHE_SHIFT ; + sbi = inode->i_sb->s_fs_info; + + kaddr =page_address (page); + + req =prepare_osd_write (sbi->s_dev, sbi->s_pid, + inode->i_ino +OSDFS_OBJ_OFF , len, start,0 , + kaddr); +if (!req) { +printk (KERN_ERR "ERROR: writepage failed.\n" ); + ret = -ENOMEM ; + goto fail; + } + + ret =osdfs_async_op (req, writepage_done, (void *)page, oi->i_cred); +if (ret) { +free_osd_req (req); + goto fail; + } +atomic_inc (&sbi->s_curr_pending); +out: +return ret; +fail: +set_bit (AS_EIO , &page->mapping->flags); +end_page_writeback (page); +unlock_page (page); + goto out; +} + +/* +* Callback for readpage +*/ +void readpage_done (struct osd_request *req,void *p) +{ +struct page *page = (struct page *)p; +struct inode *inode = page->mapping->host; +struct osdfs_sb_info *sbi = inode->i_sb->s_fs_info; +char *kaddr; +int ret; + + ret =check_ok (req); +free_osd_req (req); +atomic_dec (&sbi->s_curr_pending); + +if (ret == -EFAULT ) { + +/* In this case we were trying to read something that wasn't on +* disk yet - return a page full of zeroes. This should be OK, +* because the object should be empty (if there was a write +* before this read, the read would be waiting with the page +* locked */ + kaddr =page_address (page); +memset (kaddr,0 ,PAGE_CACHE_SIZE ); + +SetPageUptodate (page); +if (PageError (page)) +ClearPageError (page); + }else if (ret ==0 ) { + +/* Everything is OK */ +SetPageUptodate (page); +if (PageError (page)) +ClearPageError (page); + }else { + +/* Error */ +SetPageError (page); + } + +unlock_page (page); +} + +/* +* Read a page from the OSD +*/ +static int readpage_filler (struct page *page) +{ +struct osd_request *req =NULL ; +struct inode *inode = page->mapping->host; +struct osdfs_i_info *oi =OSDFS_I (inode); +ino_t ino = inode->i_ino; +loff_t i_size =i_size_read (inode); +unsigned long end_index = i_size >>PAGE_CACHE_SHIFT ; +struct super_block *sb = inode->i_sb; +struct osdfs_sb_info *sbi = sb->s_fs_info; +uint64_t amount; +unsigned char *kaddr; +int ret =0 ; + +if (!PageLocked (page)) +BUG (); + +if (PageUptodate (page)) + goto out; + +/* we are before the last page */ +if (page->index < end_index) { + amount =PAGE_CACHE_SIZE ; + goto do_it; + } + + amount = i_size & (PAGE_CACHE_SIZE -1 ); + +/* this will be out of bounds, or doesn't exist yet */ +if ((page->index >= end_index +1 || !amount) || (!ObjCreated (oi))) { + kaddr =kmap_atomic (page,KM_USER0 ); +memset (kaddr,0 ,PAGE_CACHE_SIZE ); +flush_dcache_page (page); +kunmap_atomic (page,KM_USER0 ); +SetPageUptodate (page); +if (PageError (page)) +ClearPageError (page); +unlock_page (page); + goto out; + } + +do_it: + kaddr =page_address (page); + + req =prepare_osd_read (sbi->s_dev, sbi->s_pid, ino +OSDFS_OBJ_OFF , + amount, (uint64_t )(page->index <<PAGE_CACHE_SHIFT ),0 , + kaddr); +if (!req) { +printk (KERN_ERR "ERROR: readpage failed.\n" ); + ret = -ENOMEM ; +unlock_page (page); + goto out; + } + + ret =osdfs_async_op (req, readpage_done, (void *)page, oi->i_cred); +if (ret) { +free_osd_req (req); +unlock_page (page); + goto out; + } +atomic_inc (&sbi->s_curr_pending); +out: +return ret; +} + +/* +* We don't need the file +*/ +static int osdfs_readpage (struct file *file,struct page *page) +{ +return readpage_filler (page); +} + +/* +* We don't need the data +*/ +static int readpage_strip (void *data,struct page *page) +{ +return readpage_filler (page); +} + +/* +* read a bunch of pages - usually for readahead +*/ +static int osdfs_readpages (struct file *file,struct address_space *mapping, +struct list_head *pages,unsigned nr_pages) +{ +return read_cache_pages (mapping, pages, readpage_strip,NULL ); +} + +/* This was borrowed from fs/libfs.c it used to be exported but now it +* is not. FIXME: Is this at all right? +*/ +static int osdfs_simple_commit_write (struct file *file,struct page *page, +unsigned from,unsigned to) +{ +struct inode *inode = page->mapping->host; +loff_t pos = ((loff_t )page->index <<PAGE_CACHE_SHIFT ) + to; + +if (!PageUptodate (page)) +SetPageUptodate (page); +/* +* No need to use i_size_read() here, the i_size +* cannot change under us because we hold the i_mutex. +*/ +if (pos > inode->i_size) +i_size_write (inode, pos); +set_page_dirty (page); +return 0 ; +} + +struct address_space_operations osdfs_aops = { + .readpage = osdfs_readpage, + .readpages = osdfs_readpages, + .writepage = osdfs_writepage, + .prepare_write = simple_prepare_write, + .commit_write = osdfs_simple_commit_write, + .writepages = generic_writepages, +}; +/****************************************************************************** * INODE OPERATIONS *****************************************************************************/
fs/osdfs/osdfs.h +3 -0
@@ -188,6 +188,9 @@ int osdfs_setattr(struct dentry *, struct iattr *);extern struct inode_operations osdfs_file_inode_operations;extern struct file_operations osdfs_file_operations; +/* inode.c */ +extern struct address_space_operations osdfs_aops; +/* symlink.c */ extern struct inode_operations osdfs_symlink_inode_operations;extern struct inode_operations osdfs_fast_symlink_inode_operations;
Signature · 1 line
1.6.0.1
RFC 5/9 osdfs: dir_inode and directory operations
In reply to Boaz Harroshimplementation of directory and inode operations.
* A directory is treated as a file, and essentially contains a list of <file name, inode #> pairs for files that are found in that directory. The object IDs correspond to the files' inode numbers and are allocated using a 64bit incrementing global counter.
* Each file's control block (AKA on-disk inode) is stored in its object's attributes. This applies to both regular files and other types (directories, device files, symlinks, etc.).
--- fs/osdfs/Kbuild | 2 +- fs/osdfs/dir.c | 629 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/osdfs/inode.c | 267 +++++++++++++++++++++++ fs/osdfs/namei.c | 348 ++++++++++++++++++++++++++++++ fs/osdfs/osdfs.h | 26 +++ 5 files changed, 1271 insertions(+), 1 deletions(-) create mode 100644 fs/osdfs/dir.c create mode 100644 fs/osdfs/namei.c
fs/osdfs/Kbuild +1 -1
@@ -20,5 +20,5 @@ EXTRA_CFLAGS += -I$(OSD_INC) # EXTRA_CFLAGS += -DCONFIG_OSDFS_DEBUG endif -osdfs-objs := osd.o inode.o file.o symlink.o +osdfs-objs := osd.o inode.o file.o symlink.o namei.o dir.o obj-$(CONFIG_OSDFS_FS) += osdfs.o
fs/osdfs/dir.c new file +629 -0
629 lines not on this page · raw message
fs/osdfs/inode.c +267 -0
276 lines not on this page · raw message
fs/osdfs/namei.c new file +348 -0
348 lines not on this page · raw message
fs/osdfs/osdfs.h +26 -0
@@ -106,6 +106,11 @@ static inline struct osdfs_i_info *OSDFS_I(struct inode *inode) return container_of(inode, struct osdfs_i_info, vfs_inode); } +/* +* Maximum count of links to a file +*/ +#define OSDFS_LINK_MAX 32000 +/************************* * function declarations * *************************/
@@ -179,11 +184,28 @@ void free_osd_req(struct osd_request *req);/* inode.c */ void osdfs_truncate (struct inode *inode); +extern struct inode *osdfs_iget (struct super_block *,unsigned long ); +struct inode *osdfs_new_inode (struct inode *,int );int osdfs_setattr (struct dentry *,struct iattr *); +/* dir.c: */ +int osdfs_add_link (struct dentry *,struct inode *); +ino_t osdfs_inode_by_name (struct inode *,struct dentry *); +int osdfs_delete_entry (struct osdfs_dir_entry *,struct page *); +int osdfs_make_empty (struct inode *,struct inode *); +struct osdfs_dir_entry *osdfs_find_entry (struct inode *,struct dentry *, +struct page **); +int osdfs_empty_dir (struct inode *); +struct osdfs_dir_entry *osdfs_dotdot (struct inode *,struct page **); +void osdfs_set_link (struct inode *,struct osdfs_dir_entry *,struct page *, +struct inode *); +/********************* * operation vectors * *********************/ +/* dir.c: */ +extern struct file_operations osdfs_dir_operations; +/* file.c */ extern struct inode_operations osdfs_file_inode_operations;extern struct file_operations osdfs_file_operations;
@@ -191,6 +213,10 @@ extern struct file_operations osdfs_file_operations;/* inode.c */ extern struct address_space_operations osdfs_aops; +/* namei.c */ +extern struct inode_operations osdfs_dir_inode_operations; +extern struct inode_operations osdfs_special_inode_operations; +/* symlink.c */ extern struct inode_operations osdfs_symlink_inode_operations;extern struct inode_operations osdfs_fast_symlink_inode_operations;
Signature · 1 line
1.6.0.1
RFC 6/9 osdfs: super_operations and file_system_type
In reply to Boaz HarroshThis patch ties all operation vectors into a file system superblock and registers the osdfs file_system_type at module's load time.
* The file system control block (AKA on-disk superblock) resides in an object with a special ID (defined in common.h). Information included in the file system control block is used to fill the in-memory superblock structure at mount time. This object is created before the file system is used by mkosdfs.c It contains information such as:
- The file system's magic number
- The next inode number to be allocated
--- fs/osdfs/Kbuild | 2 +- fs/osdfs/inode.c | 197 +++++++++++++++++++++- fs/osdfs/osdfs.h | 30 ++++ fs/osdfs/super.c | 502 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 729 insertions(+), 2 deletions(-) create mode 100644 fs/osdfs/super.c
fs/osdfs/Kbuild +1 -1
@@ -20,5 +20,5 @@ EXTRA_CFLAGS += -I$(OSD_INC) # EXTRA_CFLAGS += -DCONFIG_OSDFS_DEBUG endif -osdfs-objs := osd.o inode.o file.o symlink.o namei.o dir.o +osdfs-objs := osd.o inode.o file.o symlink.o namei.o dir.o super.o obj-$(CONFIG_OSDFS_FS) += osdfs.o
fs/osdfs/inode.c +196 -1
227 lines not on this page · raw message
fs/osdfs/osdfs.h +30 -0
@@ -49,6 +49,17 @@#endif /* +* struct to hold what we get from mount options +*/ +struct osdfs_mountopt { +const char *dev_name; +uint64_t pid; +int timeout; +bool mkfs; +int format;/*in Mbyte*/ +}; + +/* * our extension to the in-memory superblock */ struct osdfs_sb_info {
@@ -107,6 +118,14 @@ static inline struct osdfs_i_info *OSDFS_I(struct inode *inode) }/* +* ugly struct so that we can pass two arguments to update_inode's callback +*/ +struct updatei_args { +struct osdfs_sb_info *sbi; +struct osdfs_fcb *fcb; +}; + +/* * Maximum count of links to a file */ #define OSDFS_LINK_MAX 32000
@@ -185,9 +204,17 @@ void free_osd_req(struct osd_request *req);/* inode.c */ void osdfs_truncate (struct inode *inode);extern struct inode *osdfs_iget (struct super_block *,unsigned long ); +extern int osdfs_write_inode (struct inode *,int ); +extern void osdfs_delete_inode (struct inode *);struct inode *osdfs_new_inode (struct inode *,int );int osdfs_setattr (struct dentry *,struct iattr *); +/* super.c: */ +#ifdef OSDFS_DEBUG +void osdfs_dprint_internal (char *str, ...); +#endif +extern void osdfs_write_super (struct super_block *); +/* dir.c: */ int osdfs_add_link (struct dentry *,struct inode *);ino_t osdfs_inode_by_name (struct inode *,struct dentry *);
@@ -217,6 +244,9 @@ extern struct address_space_operations osdfs_aops;extern struct inode_operations osdfs_dir_inode_operations;extern struct inode_operations osdfs_special_inode_operations; +/* super.c */ +extern struct super_operations osdfs_sops; +/* symlink.c */ extern struct inode_operations osdfs_symlink_inode_operations;extern struct inode_operations osdfs_fast_symlink_inode_operations;
fs/osdfs/super.c new file +502 -0
502 lines not on this page · raw message
Signature · 1 line
1.6.0.1
RFC 7/9 osdfs: mkosdfs
In reply to Boaz HarroshWe need a mechanism to prepare the file system (mkfs). I chose to implement that by means of a couple of mount-options. Because there is no user-mode API for committing OSD commands. And also, all this stuff is highly internal to the file system itself.
- Added two mount options mkfs=0/1,format=capacity_in_meg, so mkfs/format can be executed by kernel code just before mount. An mkosdfs utility can now be implemented by means of a script that mounts and unmount the file system with proper options.
--- fs/osdfs/Kbuild | 2 +- fs/osdfs/mkosdfs.c | 605 ++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/osdfs/osdfs.h | 3 + fs/osdfs/super.c | 18 ++ 4 files changed, 627 insertions(+), 1 deletions(-) create mode 100644 fs/osdfs/mkosdfs.c
fs/osdfs/Kbuild +1 -1
@@ -20,5 +20,5 @@ EXTRA_CFLAGS += -I$(OSD_INC) # EXTRA_CFLAGS += -DCONFIG_OSDFS_DEBUG endif -osdfs-objs := osd.o inode.o file.o symlink.o namei.o dir.o super.o +osdfs-objs := osd.o inode.o file.o symlink.o namei.o dir.o super.o mkosdfs.o obj-$(CONFIG_OSDFS_FS) += osdfs.o
fs/osdfs/mkosdfs.c new file +605 -0
605 lines not on this page · raw message
fs/osdfs/osdfs.h +3 -0
@@ -201,6 +201,9 @@ int extract_list_from_req(struct osd_request *req,void free_osd_req (struct osd_request *req); +/* mkosdfs.c */ +int osdfs_mkfs (struct osd_dev *dev,uint64_t p_id,uint64_t format_size); +/* inode.c */ void osdfs_truncate (struct inode *inode);extern struct inode *osdfs_iget (struct super_block *,unsigned long );
fs/osdfs/super.c +18 -0
@@ -55,6 +55,8 @@ enum { Opt_lun, Opt_tid, Opt_pid, Opt_to, Opt_mkfs, Opt_format, Opt_err };static match_table_t tokens = { {Opt_pid,"pid=%u" }, {Opt_to,"to=%u" }, + {Opt_mkfs,"mkfs=%u" }, + {Opt_format,"format=%u" }, {Opt_err,NULL } };
@@ -100,6 +102,16 @@ static int parse_options(char *options, struct osdfs_mountopt *opts) } opts->timeout = option;break ; +case Opt_mkfs: +if (match_int (&args[0 ], &option)) +return -EINVAL ; + opts->mkfs = option !=0 ; +break ; +case Opt_format: +if (match_int (&args[0 ], &option)) +return -EINVAL ; + opts->format = option; +break ; } }
@@ -277,6 +289,12 @@ static int osdfs_fill_super(struct super_block *sb, void *data, int silent) sb->s_bdev =NULL ; sb->s_dev =0 ; +/* see if we need to make the file system on the obsd */ +if (opts->mkfs) { +OSDFS_DBGMSG ("osdfs_mkfs %p\n" , sbi->s_dev); +osdfs_mkfs (sbi->s_dev, sbi->s_pid, opts->format); + } +/* read data from on-disk superblock object */ make_credential (sbi->s_cred, sbi->s_pid,OSDFS_SUPER_ID );
Signature · 1 line
1.6.0.1
RFC 8/9 osdfs: Documentation
In reply to Boaz HarroshAdded some documentation in osdfs.txt, as well as a BUGS file.
For further reading, operation instructions, example scripts and up to date infomation and code please see:
http://open-osd.org
--- Documentation/filesystems/osdfs.txt | 173 +++++++++++++++++++++++++++++++++++ fs/osdfs/BUGS | 6 + 2 files changed, 179 insertions(+), 0 deletions(-) create mode 100644 Documentation/filesystems/osdfs.txt create mode 100644 fs/osdfs/BUGS
Documentation/filesystems/osdfs.txt new file +173 -0
@@ -0,0 +1,173 @@ +=============================================================================== +WHAT IS OSDFS? +=============================================================================== + +osdfs is a file system that uses an OSD and exports the API of a normal Linux +file system. Users access osdfs like any other local file system, and osdfs +will in turn issue commands to the local initiator. + +=============================================================================== +ENVIRONMENT +=============================================================================== + +To use this file system, you need to have an object store to run it on. You +may download a target from: +http://open-osd.org + +See drivers/scsi/osd/README for how to setup a working osd environment. + +=============================================================================== +USAGE +=============================================================================== + +1. Download and compile osdfs and open-osd initiator: + You need an external Kernel source tree or kernel headers from your + distribution. (anything based on 2.6.26 or later). + + a. download open-osd including osdfs source using: + [parent-directory]$ git clone git://git.open-osd.org/open-osd.git + + b. Build the library module like this: + [parent-directory]$ make -C KSRC=$(KER_DIR) open-osd + + This will build both the open-osd initiator as well as the osdfs kernel + module. Use whatever parameters you compiled your Kernel with and + $(KER_DIR) above pointing to the Kernel you compile against. See the file + open-osd/top-level-Makefile for an example. + +2. Get the OSD initiator and target set up properly, and login to the target. + See drivers/scsi/osd/README for farther instructions. Also see ./do-osd-test + for example script that does all these steps. + +3. Insmod the osdfs.ko module: + [osdfs]$ insmod osdfs.ko + +4. Make sure the directory where you want to mount exists. If not, create it. + (For example, /mnt/osdfs) + +5. At first run you will need to invoke the mkosdfs.c routine + + As an example, this will create the file system on: + /dev/osd0 partition ID 65540, max capacity 1024 Mg bytes + + mount -t osdfs -o pid=65540,mkfs=1,format=1024 /dev/osd0 /mnt/osdfs/ + + The format=1024 is optional if not specified no OSD_FORMAT will be preformed + and a clean file system will be created in the specified pid, in the + available space of the target. + If pid already exist it will be deleted and a new one will be created in it's + place. Be careful. + +6. Mount the file system. The above command left the filesystem mounted, + but on subsequent runs the mkfs=1 should not be invoked. + + For example, to mount /dev/osd0, partition ID 65540 on /mnt/osdfs: + + mount -t osdfs -o pid=65540 /dev/osd0 /mnt/osdfs/ + +7. For reference (under fs/osdfs/): + do-osdfs start - an example of how to perform the above steps. + do-osdfs stop - an example of how to unmount the file system. + +8. Extra compilation flags (uncomment in fs/osdfs/Kbuild): + OSDFS_DEBUG - for debug messages and extra checks. + +=============================================================================== +osdfs mount options +=============================================================================== +Similar to any mount command: + mount -t osdfs -o osdfs_options /dev/osdX mount_osdfs_directory + +Where: + -t osdfs: specifies the osdfs file system + + /dev/osdX: X is a decimal number. /dev/osdX was created after a successful + login into an OSD target. + + mount_osdfs_directory: The directory to mount the file system on + + osdfs_options: Options are separated by commas (,) + pid=<integer> - The partition number to mount/create as + container of the filesystem. + This option is mandatory + mkfs=<1/0> - If mkfs=1 make a new filesystem before mount. + Default is 0 - don't make. If mkfs=0 pid must + exist and an mkfs=1 was previously preformed + on it. + format=<integer>- If mkfs=1 is specified then the format= + parameter will also invoke an OSD_FORMAT + command prior to creation of the filesystem + partition (mkfs). The integer specified is in + Mega bytes. If not specified or set to 0 then + no format is executed, and a partition is + created in the available space. + If mkfs=0 this option is ignored. + to=<integer> - Timeout in ticks for a single command + default is (60 * HZ) [for debugging only] + +=============================================================================== +DESIGN +=============================================================================== + +* The file system control block (AKA on-disk superblock) resides in an object + with a special ID (defined in common.h). + Information included in the file system control block is used to fill the + in-memory superblock structure at mount time. This object is created before + the file system is used by mkosdfs.c It contains information such as: + - The file system's magic number + - The next inode number to be allocated + +* Each file resides in its own object and contains the data (and it will be + possible to extend the file over multiple objects, though this has not been + implemented yet). + +* A directory is treated as a file, and essentially contains a list of <file + name, inode #> pairs for files that are found in that directory. The object + IDs correspond to the files' inode numbers and will be allocated according to + a bitmap (stored in a separate object). Now they are allocated using a + counter. + +* Each file's control block (AKA on-disk inode) is stored in its object's + attributes. This applies to both regular files and other types (directories, + device files, symlinks, etc.). + +* Credentials are generated per object (inode and superblock) when they is + created in memory (read off disk or created). The credential works for all + operations and is used as long as the object remains in memory. + +* Async OSD operations are used whenever possible, but the target may execute + them out of order. The operations that concern us are create, delete, + readpage, writepage, update_inode, and truncate. The following pairs of + operations should execute in the order written, and we need to prevent them + from executing in reverse order: + - The following are handled with the OBJ_CREATED and OBJ_2BCREATED + flags. OBJ_CREATED is set when we know the object exists on the OSD - + in create's callback function, and when we successfully do a read_inode. + OBJ_2BCREATED is set in the beginning of the create function, so we + know that we should wait. + - create/delete: delete should wait until the object is created + on the OSD. + - create/readpage: readpage should be able to return a page + full of zeroes in this case. If there was a write already + en-route (i.e. create, writepage, readpage) then the page + would be locked, and so it would really be the same as + create/writepage. + - create/writepage: if writepage is called for a sync write, it + should wait until the object is created on the OSD. + Otherwise, it should just return. + - create/truncate: truncate should wait until the object is + created on the OSD. + - create/update_inode: update_inode should wait until the + object is created on the OSD. + - Handled by VFS locks: + - readpage/delete: shouldn't happen because of page lock. + - writepage/delete: shouldn't happen because of page lock. + - readpage/writepage: shouldn't happen because of page lock. + +=============================================================================== +LICENSE/COPYRIGHT +=============================================================================== +The osdfs file system is based on ext2 v0.5b (distributed with the Linux kernel +version 2.6.10). All files include the original copyrights, and the license +is GPL version 2 (only version 2, as is true for the Linux kernel). The +Linux kernel can be downloaded from www.kernel.org.
fs/osdfs/BUGS new file +6 -0
@@ -0,0 +1,6 @@ +- Some mount time options should have been 64-bit, but are declared as 32-bit + because that's what the kernel's parsing methods support at this time. + +- Out-of-space may cause a severe problem if the object (and directory entry) + were written, but the inode attributes failed. Then if the filesystem was + unmounted and mounted the kernel can get into an endless loop doing a readdir.
Signature · 1 line
1.6.0.1
RFC 9/9 open-osd: Global Makefile and do-osdfs test script
In reply to Boaz HarroshAdded a global out-of-tree Makefile that includes both fs/osdfs and drivers/scsi/osd. This way imported symbols from libosd are accounted for by the Kernel's build system
Added a small interactive script to test drive osdfs mount/unmout and preform some tests.
--- Makefile | 20 ++++++++++++ do-osdfs | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 126 insertions(+), 0 deletions(-) create mode 100644 Makefile create mode 100755 do-osdfs
Makefile new file +20 -0
@@ -0,0 +1,20 @@ +OSD_INC=`pwd`/include +LIBOSD=drivers/scsi/osd +OSDFS=fs/osdfs + +# Kbuild part (Embeded in this Makefile) +obj-m := $(LIBOSD)/ $(OSDFS)/ + +# Makefile for out-of-tree builds +KSRC ?= /lib/modules/$(shell uname -r)/build +KBUILD_OUTPUT ?= +ARCH ?= + +# this is the basic Kbuild out-of-tree invocation, with the M= option +KBUILD_BASE = +$(MAKE) -C $(KSRC) M=`pwd` KBUILD_OUTPUT=$(KBUILD_OUTPUT) ARCH=$(ARCH) + +all: ; + $(KBUILD_BASE) OSD_INC=$(OSD_INC) modules + +clean: ; + $(KBUILD_BASE) clean
do-osdfs new file +106 -0
106 lines not on this page · raw message
Signature · 1 line
1.6.0.1
Quoted 35 lines · Boaz Harrosh
Please review an OSD based file system.
Given that our OSD initiator library is accepted into Kernel, we would like to also submit an osdfs. This is the first iteration of this file system.
The next stage is to make it exportable by the pNFS-over-objects Server. osdfs is one of the building blocks for a full, end-to-end open source reference implementation of a Server/Client pNFS-over-objects we want to have available in Linux. Other parts are the Generic pNFS client project with the objects-layout-driver, and the generic pNFS server plus osdfs once it is adapted to be exportable.
(See all about pNFS in Linux at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_prototype_design)osdfs was originally developed by Avishay Traeger <[email protected]> from IBM. A very old version of it is hosted on sourceforge as the osdfs project. It was originally developed for the 2.6.10 Kernel over the old IBM's osd-initiator Linux driver.
Since then it was picked by us, open-osd, and was both forward ported to current Kernel, as well as converted to run over our osd Kernel Library. The conversion effort, if anyone is interested, is also available as a patchset here:
git-clonegit://git-open-osd.org/open-osd.gitosdfs-devel
or on the web at:
http://git.open-osd.org/gitweb.cgi?p=open-osd.git;a=shortlog;h=refs/heads/osdfs-develThe Original code is based on ext2 code from the Kernel at the time. Further reading is available at the last patch in the osdfs.txt file.
I have mechanically divided the code in parts, each introducing a group of vfs function vectors, all tied at the end into a full filesystem. Each patch can be compiled but it will only run at the very end. This was done for the hope of easier reviewing.
Quoted 9 lines
Here is the list of patches
[RFC 1/9] osdfs: osd Swiss army knife
[RFC 2/9] osdfs: file andfile_inodeoperations
[RFC 3/9] osdfs:symlink_inodeandfast_symlink_inodeoperations [RFC 4/9] osdfs:address_space_operations
[RFC 5/9] osdfs:dir_inodeand directory operations [RFC 6/9] osdfs:super_operationsandfile_system_type[RFC 7/9] osdfs: mkosdfs
[RFC 8/9] osdfs: Documentation
Pretty cool stuff.
I've been wondering when we would start seeing OSD filesystems make their appearance.
Random, unordered comments:
* This is important stuff. Should have been posted to LKML. Please CC LKML in the future.
* As discussed at the filesystem summit, OSD use implies a need for an MD-like layer for OSD objects. Has anyone even started the design work for this?
* I tend to think there is room for more than one OSD filesystem in the Linux kernel. Assuming all OSDs will use the same Linux filesystem driver will lead to bloat, and you potentially "code yourself into a corner." Let's not rule out multiple filesystems.
As such, "osdfs" seems like too-generic a name. How about boazfs? :)
* Get this into the kernel ASAP! OSD stuff languishes outside the kernel for _far_ too long. OSD is a key storage technology that needs to be developed in the full light of the Linux community, not off in a dark corner somewhere, where few see progress or discussions.
Object-based storage, and its SCSI incarnation OSD, is a MAJOR revision of the block storage API, moving away from LBA-addressed linear APIs. That's a big deal, and should be discussed on LKML, IMO...
Jeff
Quoted 51 lines · Jeff Garzik
Quoted 35 lines · Boaz Harrosh
Please review an OSD based file system.
Given that our OSD initiator library is accepted into Kernel, we would like to also submit an osdfs. This is the first iteration of this file system.
The next stage is to make it exportable by the pNFS-over-objects Server. osdfs is one of the building blocks for a full, end-to-end open source reference implementation of a Server/Client pNFS-over-objects we want to have available in Linux. Other parts are the Generic pNFS client project with the objects-layout-driver, and the generic pNFS server plus osdfs once it is adapted to be exportable.
(See all about pNFS in Linux at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_prototype_design)osdfs was originally developed by Avishay Traeger <[email protected]> from IBM. A very old version of it is hosted on sourceforge as the osdfs project. It was originally developed for the 2.6.10 Kernel over the old IBM's osd-initiator Linux driver.
Since then it was picked by us, open-osd, and was both forward ported to current Kernel, as well as converted to run over our osd Kernel Library. The conversion effort, if anyone is interested, is also available as a patchset here:
git-clonegit://git-open-osd.org/open-osd.gitosdfs-devel
or on the web at:
http://git.open-osd.org/gitweb.cgi?p=open-osd.git;a=shortlog;h=refs/heads/osdfs-develThe Original code is based on ext2 code from the Kernel at the time. Further reading is available at the last patch in the osdfs.txt file.
I have mechanically divided the code in parts, each introducing a group of vfs function vectors, all tied at the end into a full filesystem. Each patch can be compiled but it will only run at the very end. This was done for the hope of easier reviewing.
Quoted 9 lines
Here is the list of patches
[RFC 1/9] osdfs: osd Swiss army knife
[RFC 2/9] osdfs: file andfile_inodeoperations
[RFC 3/9] osdfs:symlink_inodeandfast_symlink_inodeoperations [RFC 4/9] osdfs:address_space_operations
[RFC 5/9] osdfs:dir_inodeand directory operations [RFC 6/9] osdfs:super_operationsandfile_system_type[RFC 7/9] osdfs: mkosdfs
[RFC 8/9] osdfs: DocumentationPretty cool stuff.
I've been wondering when we would start seeing OSD filesystems make their appearance.
Quoted 8 lines
Random, unordered comments:
* This is important stuff. Should have been posted to LKML. Please CC LKML in the future.
* As discussed at the filesystem summit, OSD use implies a need for an MD-like layer for OSD objects. Has anyone even started the design work for this?
Yes. I have.
I'm coding a prototype to be used by both this file system and by the pnfs objects layout driver.
Initially it will do striping and mirroring, and RAID-5 parity as a stretched goal for the initial release.
Quoted 7 lines
* I tend to think there is room for more than one OSD filesystem in the Linux kernel. Assuming all OSDs will use the same Linux filesystem driver will lead to bloat, and you potentially "code yourself into a corner." Let's not rule out multiple filesystems.
As such, "osdfs" seems like too-generic a name. How about boazfs? :)
I agree. osdfs is the name given by Avishay and IBM and we just adopted it. I think that obfs (Object-based File System) would better represent what it is (although it's still generic compared to boazfs :-)
Quoted 5 lines
* Get this into the kernel ASAP! OSD stuff languishes outside the kernel for _far_ too long. OSD is a key storage technology that needs to be developed in the full light of the Linux community, not off in a dark corner somewhere, where few see progress or discussions.
I completely agree. We've missed the merge window for 2.6.28 but if we can get it into 2.6.29 that would be great!
Quoted 4 lines
Object-based storage, and its SCSI incarnation OSD, is a MAJOR revision of the block storage API, moving away from LBA-addressed linear APIs. That's a big deal, and should be discussed on LKML, IMO...
Absolutely.
Thanks for your comments!
Benny
Quoted 3 lines
Jeff
Quoted 60 lines · Benny Halevy
Quoted 50 lines · Jeff Garzik
Quoted 35 lines · Boaz Harrosh
Please review an OSD based file system.
Given that our OSD initiator library is accepted into Kernel, we would like to also submit an osdfs. This is the first iteration of this file system.
The next stage is to make it exportable by the pNFS-over-objects Server. osdfs is one of the building blocks for a full, end-to-end open source reference implementation of a Server/Client pNFS-over-objects we want to have available in Linux. Other parts are the Generic pNFS client project with the objects-layout-driver, and the generic pNFS server plus osdfs once it is adapted to be exportable.
(See all about pNFS in Linux at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_prototype_design)osdfs was originally developed by Avishay Traeger <[email protected]> from IBM. A very old version of it is hosted on sourceforge as the osdfs project. It was originally developed for the 2.6.10 Kernel over the old IBM's osd-initiator Linux driver.
Since then it was picked by us, open-osd, and was both forward ported to current Kernel, as well as converted to run over our osd Kernel Library. The conversion effort, if anyone is interested, is also available as a patchset here:
git-clonegit://git-open-osd.org/open-osd.gitosdfs-devel
or on the web at:
http://git.open-osd.org/gitweb.cgi?p=open-osd.git;a=shortlog;h=refs/heads/osdfs-develThe Original code is based on ext2 code from the Kernel at the time. Further reading is available at the last patch in the osdfs.txt file.
I have mechanically divided the code in parts, each introducing a group of vfs function vectors, all tied at the end into a full filesystem. Each patch can be compiled but it will only run at the very end. This was done for the hope of easier reviewing.
Quoted 9 lines
Here is the list of patches
[RFC 1/9] osdfs: osd Swiss army knife
[RFC 2/9] osdfs: file andfile_inodeoperations
[RFC 3/9] osdfs:symlink_inodeandfast_symlink_inodeoperations [RFC 4/9] osdfs:address_space_operations
[RFC 5/9] osdfs:dir_inodeand directory operations [RFC 6/9] osdfs:super_operationsandfile_system_type[RFC 7/9] osdfs: mkosdfs
[RFC 8/9] osdfs: DocumentationPretty cool stuff.
I've been wondering when we would start seeing OSD filesystems make their appearance.
Quoted 8 lines
Random, unordered comments:
* This is important stuff. Should have been posted to LKML. Please CC LKML in the future.
* As discussed at the filesystem summit, OSD use implies a need for an MD-like layer for OSD objects. Has anyone even started the design work for this?
Quoted 6 lines
Yes. I have.
I'm coding a prototype to be used by both this file system and by the pnfs objects layout driver.
Initially it will do striping and mirroring, and RAID-5 parity as a stretched goal for the initial release.
Thanks Benny, it would be nice to connect all these thing together.
Quoted 11 lines
Quoted 6 lines
* I tend to think there is room for more than one OSD filesystem in the Linux kernel. Assuming all OSDs will use the same Linux filesystem driver will lead to bloat, and you potentially "code yourself into a corner." Let's not rule out multiple filesystems.
As such, "osdfs" seems like too-generic a name. How about boazfs? :)
I agree. osdfs is the name given by Avishay and IBM and we just adopted it. I think that obfs (Object-based File System) would better represent what it is (although it's still generic compared to boazfs :-)
If at all then it's avishayfs, but unless I write this filesystem from scratch I don't think I can do anything about the name. The code is copyrighted to Avishay Traeger, and that is the name he chose. Also he has a sourceforge.net project of that name for a long time.
Avishay would you be willing to change the name? See above, people think it is too generic a name. Like if someone would do a scsifs or blocksfs.
Personally for me it's just a name, I don't mind either way.
Quoted 8 lines
Quoted 4 lines
* Get this into the kernel ASAP! OSD stuff languishes outside the kernel for _far_ too long. OSD is a key storage technology that needs to be developed in the full light of the Linux community, not off in a dark corner somewhere, where few see progress or discussions.
I completely agree. We've missed the merge window for 2.6.28 but if we can get it into 2.6.29 that would be great!
Quoted 7 lines
Quoted 3 lines
Object-based storage, and its SCSI incarnation OSD, is a MAJOR revision of the block storage API, moving away from LBA-addressed linear APIs. That's a big deal, and should be discussed on LKML, IMO...
Absolutely.
Thanks for your comments!
I've been working on this OSD stuff for a while now, and I'm very excited about it, it feels very powerful yet simple. I was able to reach stable results (hopefully) in relatively short time. and the code size of both the Initiator and the osdfs is pretty small.
I forgot to mention in my introduction that I was able to clone a git tree over an osdfs mount, compile a kernel, which actually runs. Make changes git-diff and commit the changes. Unmount/mount and diff with original tree with success. So it is functional. With Benny's stuff it might even get fast with the right setup.
Quoted 4 lines
Benny
Quoted 2 lines
Jeff
Boaz
On Tue, Nov 4, 2008 at 12:11 PM, Boaz Harrosh <[email protected]> wrote:
<snip>
Quoted 12 lines
Quoted 6 lines
Quoted 1 line
As such, "osdfs" seems like too-generic a name. How about boazfs? :)
I agree. osdfs is the name given by Avishay and IBM and we just adopted it. I think that obfs (Object-based File System) would better represent what it is (although it's still generic compared to boazfs :-)
If at all then it's avishayfs, but unless I write this filesystem from scratch I don't think I can do anything about the name. The code is copyrighted to Avishay Traeger, and that is the name he chose. Also he has a sourceforge.net project of that name for a long time.
Quoted 5 lines
Avishay would you be willing to change the name? See above, people think it is too generic a name. Like if someone would do a scsifs or blocksfs.
Personally for me it's just a name, I don't mind either way.
I don't care if we change the name. I suppose we can discuss alternatives outside of the mailing lists, but I'd rather not do boazfs or avishayfs :-)
By the way, just to clarify: I wrote this file system at IBM, but they were nice enough to give me full ownership of the code. I then proceeded to put the code up on sourceforge. Thanks to Boaz and Benny for reviving the code and submitting it for inclusion in the kernel.
Avishay