exofs: few patches for Linux 2.6.31
Subsystems (3)
cover exofs: few patches for Linux 2.6.31
For review the few exofs patches I would like included for current merge window (Linux 2.6.31)
The main effort on the exofs front was to make it pnfs-exportable. Which for now only lives in the pnfs tree. So there was not much left outside of that.
Also for final review is the osdblk stacking block driver which is ready for submission but is missing a supporting user-mode tool, and some more testing love.
The list of patches:
[PATCH 1/6] exofs: Fix bio leak in error handling path (sync read)
A small and rare bug fix
[PATCH 2/6] exofs: Remove IBM copyrights
As requested by Original author and IBM folks. The code does not belong to IBM. (Never was)
[PATCH 3/6] exofs: Avoid using file_fsync()
This one is left like this. Since other options suggested are not good for exofs since it does not have a block-device and any of the BH stuff.
[PATCH 4/6] MAINTAINERS: Add osd maintained files (F:)
Use the new MAINTAINERS F: annotation for the OSD files
[PATCH 5/6] osdblk: a Linux block device for OSD objects [PATCH 6/6] osdblk: Adjust queue limits to lower device's limits
This is the proposed osdblk driver by Jeff Garzik. It as all the Kernel pre-requisites, but is missing a user-mode tool and more testing. So I'm not sure it will make it into this Kernel. But for review.
Thanks
Boaz
PATCH 1/6 exofs: Fix bio leak in error handling path (sync read)
When failing a read request in the sync path, called from write_begin, I forgot to free the allocated bio, fix it.
--- fs/exofs/inode.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
fs/exofs/inode.c +3 -0
@@ -295,6 +295,9 @@ static int read_exec(struct page_collect *pcol, bool is_sync) err:if (!is_sync)_unlock_pcol_pages (pcol, ret,READ ); +else /* Pages unlocked by caller in sync mode only free bio */ +pcol_free (pcol); +kfree (pcol_copy);if (or)osd_end_request (or);
Signature · 1 line
1.6.2.1
PATCH 2/6 exofs: Remove IBM copyrights
In reply to Boaz HarroshBoaz,
Congrats on getting all the OSD stuff into 2.6.30!
I just pulled the git, and saw that the IBM copyrights are still there. Please remove them from all files:
* Copyright (C) 2005, 2006
* International Business Machines
IBM has revoked all rights on the code - they gave it to me.
Thanks!
Avishay
--- fs/exofs/common.h | 4 +--- fs/exofs/dir.c | 4 +--- fs/exofs/exofs.h | 4 +--- fs/exofs/file.c | 4 +--- fs/exofs/inode.c | 4 +--- fs/exofs/namei.c | 4 +--- fs/exofs/osd.c | 4 +--- fs/exofs/super.c | 4 +--- fs/exofs/symlink.c | 4 +--- 9 files changed, 9 insertions(+), 27 deletions(-)
fs/exofs/common.h +1 -3
@@ -2,9 +2,7 @@* common.h - Common definitions for both Kernel and user-mode utilities * * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
fs/exofs/dir.c +1 -3
@@ -1,8 +1,6 @@/* * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
fs/exofs/exofs.h +1 -3
@@ -1,8 +1,6 @@/* * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
fs/exofs/file.c +1 -3
@@ -1,8 +1,6 @@/* * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
fs/exofs/inode.c +1 -3
@@ -1,8 +1,6 @@/* * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
fs/exofs/namei.c +1 -3
@@ -1,8 +1,6 @@/* * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
fs/exofs/osd.c +1 -3
@@ -1,8 +1,6 @@/* * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
fs/exofs/super.c +1 -3
@@ -1,8 +1,6 @@/* * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
fs/exofs/symlink.c +1 -3
@@ -1,8 +1,6 @@/* * Copyright (C) 2005, 2006 -* Avishay Traeger ([email protected]) ([email protected]) -* Copyright (C) 2005, 2006 -* International Business Machines +* Avishay Traeger ([email protected]) * Copyright (C) 2008, 2009 * Boaz Harrosh <[email protected]> *
Signature · 1 line
1.6.2.1
PATCH 3/6 exofs: Avoid using file_fsync()
In reply to Boaz HarroshThe use of file_fsync() in exofs_file_sync() is not necessary since it does some extra stuff not used by exofs. Open code just the parts that are currently needed.
TODO: Farther optimization can be done to sync the sb only on inode update of new files, Usually the sb update is not needed in exofs.
--- fs/exofs/exofs.h | 3 +++ fs/exofs/file.c | 17 ++++++++++++----- fs/exofs/super.c | 2 +- 3 files changed, 16 insertions(+), 6 deletions(-)
fs/exofs/exofs.h +3 -0
@@ -154,6 +154,9 @@ ino_t exofs_parent_ino(struct dentry *child);int exofs_set_link (struct inode *,struct exofs_dir_entry *,struct page *,struct inode *); +/* super.c */ +int exofs_sync_fs (struct super_block *sb,int wait); +/********************* * operation vectors * *********************/
fs/exofs/file.c +12 -5
@@ -45,16 +45,23 @@ static int exofs_file_fsync(struct file *filp, struct dentry *dentry, {int ret;struct address_space *mapping = filp->f_mapping; +struct inode *inode = dentry->d_inode; +struct super_block *sb; ret =filemap_write_and_wait (mapping);if (ret)return ret; -/*Note: file_fsync below also calles sync_blockdev, which is a no-op -* for exofs, but other then that it does sync_inode and -* sync_superblock which is what we need here. -*/ -return file_fsync (filp, dentry, datasync); +/* sync the inode attributes */ + ret =write_inode_now (inode,1 ); + +/* This is a good place to write the sb */ +/* TODO: Sechedule an sb-sync on create */ + sb = inode->i_sb; +if (sb->s_dirt) +exofs_sync_fs (sb,1 ); + +return ret; }static int exofs_flush (struct file *file,fl_owner_t id)
fs/exofs/super.c +1 -1
@@ -198,7 +198,7 @@ static const struct export_operations exofs_export_ops;/* * Write the superblock to the OSD */ -static int exofs_sync_fs (struct super_block *sb,int wait) +int exofs_sync_fs (struct super_block *sb,int wait) {struct exofs_sb_info *sbi;struct exofs_fscb *fscb;
Signature · 1 line
1.6.2.1
PATCH 4/6 MAINTAINERS: Add osd maintained files (F:)
In reply to Boaz HarroshOSD files are found in three places:drivers/scsi/osd/include/scsi/osd_*fs/exofs/
--- MAINTAINERS | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
MAINTAINERS +3 -0
@@ -4331,6 +4331,9 @@ L: [email protected] W: http://open-osd.org T: git git://git.open-osd.org/open-osd.git S: Maintained +F: drivers/scsi/osd/ +F: drivers/include/scsi/osd_* +F: fs/exofs/ P54 WIRELESS DRIVER P: Michael Wu
Signature · 1 line
1.6.2.1
PATCH 5/6 osdblk: a Linux block device for OSD objects
In reply to Boaz HarroshFrom: Jeff Garzik <[email protected]>
Submitted driver exports a block device of the form /dev/osdblkX, where X is a decimal number.
It does that by mounting a stacking block device on top of an osd object. For example, if you create a 2G object on an OSD device, you can then use this module to present that 2G object as a Linux block device.
See inside patch for exact documentation.
[Sitting at linux-next helped fix proper Kconfig dependency
for this driver, thanks to Randy Dunlap]
--- drivers/block/Kconfig | 16 ++ drivers/block/Makefile | 1 + drivers/block/osdblk.c | 663 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 680 insertions(+), 0 deletions(-) create mode 100644 drivers/block/osdblk.c
drivers/block/Kconfig +16 -0
@@ -298,6 +298,22 @@ config BLK_DEV_NBD If unsure, say N. +config BLK_DEV_OSD + tristate "OSD object-as-blkdev support" + depends on SCSI_OSD_ULD + ---help--- + Saying Y or M here will allow the exporting of a single SCSI + OSD (object-based storage) object as a Linux block device. + + For example, if you create a 2G object on an OSD device, + you can then use this module to present that 2G object as + a Linux block device. + + To compile this driver as a module, choose M here: the + module will be called osdblk. + + If unsure, say N. + config BLK_DEV_SX8 tristate "Promise SATA SX8 support" depends on PCI
drivers/block/Makefile +1 -0
@@ -23,6 +23,7 @@ obj-$(CONFIG_XILINX_SYSACE) += xsysace.o obj-$(CONFIG_CDROM_PKTCDVD) += pktcdvd.o obj-$(CONFIG_MG_DISK) += mg_disk.o obj-$(CONFIG_SUNVDC) += sunvdc.o +obj-$(CONFIG_BLK_DEV_OSD) += osdblk.o obj-$(CONFIG_BLK_DEV_UMEM) += umem.o obj-$(CONFIG_BLK_DEV_NBD) += nbd.o
drivers/block/osdblk.c new file +663 -0
663 lines not on this page · raw message
Signature · 1 line
1.6.2.1
PATCH 6/6 osdblk: Adjust queue limits to lower device's limits
In reply to Boaz Harroshcall blk_queue_stack_limits() to copy queue limits from the underline osd scsi_device. This is absolutely needed because osdblk cannot sleep when allocating a lower-request and therefore cannot be bouncing.
TODO: Dynamic changes of limits to the lower device queue will not reflect in the upper driver
--- drivers/block/osdblk.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-)
drivers/block/osdblk.c +7 -0
@@ -66,6 +66,7 @@#include <scsi/osd_initiator.h> #include <scsi/osd_attributes.h> #include <scsi/osd_sec.h> +#include <scsi/scsi_device.h> #define DRV_NAME "osdblk"#define PFX DRV_NAME ": "
@@ -410,6 +411,12 @@ static int osdblk_init_disk(struct osdblk_device *osdev)return rc; } +/* Set our limits to the lower device limits, because osdblk cannot +* sleep when allocating a lower-request and therefore cannot be +* bouncing. +*/ +blk_queue_stack_limits (q,osd_request_queue (osdev->osd)); +blk_queue_prep_rq (q, blk_queue_start_tag);blk_queue_ordered (q,QUEUE_ORDERED_DRAIN_FLUSH , osdblk_prepare_flush);
Signature · 1 line
1.6.2.1
Quoted 8 lines · Boaz Harrosh
[PATCH 5/6] osdblk: a Linux block device for OSD objects [PATCH 6/6] osdblk: Adjust queue limits to lower device's limits
This is the proposed osdblk driver by Jeff Garzik. It as all the Kernel pre-requisites, but is missing a user-mode tool and more testing. So I'm not sure it will make it into this Kernel. But for review.
Jeff hi.
I've hacked up a very quick and dirty small utility for Creating / Removing / Resizing objects on an OSD device. [I'm sending as reply to this mail]
Please give it a fast testing. Should we now submit the osdblk driver, for 2.6.31? Which tree? I can push it through the open-osd.org tree. The driver was included for some weeks in linux-next through that tree.
Thanks
Boaz
PATCH 1/1 open-osd: osdblk User Mode utility
A minimal user-mode application to Create / Remove / Resize OSD objects from a device, for use with the osdblk.ko block device driver.
See inside patch for Usage instructions.
--- usr/Makefile | 12 ++- usr/osdblk.c | 336 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 347 insertions(+), 1 deletions(-) create mode 100644 usr/osdblk.c
usr/Makefile +11 -1
@@ -36,7 +36,7 @@ CFLAGS = -fPIC $(CWARN) $(INCLUDES) $(DEFINES) OSD_LIBS=-L../lib -losd -ALL = osd_test mkfs.exofs +ALL = osd_test mkfs.exofs osdblk all: $(DEPEND) $(ALL) clean: $(ALL:=_clean)
@@ -78,6 +78,16 @@ mkfs.exofs_clean: $(DEPEND): $(mkfs_COMMON_OBJ:.o=.c) $(mkfs_OBJ:.o=.c) +# =============== osdblk ====================================================== +osdblk_OBJ=osdblk.o + +osdblk: $(osdblk_OBJ) + $(CC) -o $@ $^ $(OSD_LIBS) + +osdblk_clean: + +$(DEPEND): $(osdblk_OBJ:.o=.c) + # =============== common rules ================================================= # every thing should compile if Makefile changed %.o: %.c Makefile
usr/osdblk.c new file +336 -0
336 lines not on this page · raw message
Signature · 1 line
1.6.2.1
Quoted 9 lines · Boaz Harrosh
Quoted 7 lines · Boaz Harrosh
[PATCH 5/6] osdblk: a Linux block device for OSD objects [PATCH 6/6] osdblk: Adjust queue limits to lower device's limits
This is the proposed osdblk driver by Jeff Garzik. It as all the Kernel pre-requisites, but is missing a user-mode tool and more testing. So I'm not sure it will make it into this Kernel. But for review.
Quoted 4 lines
Jeff hi.
I've hacked up a very quick and dirty small utility for Creating / Removing / Resizing objects on an OSD device. [I'm sending as reply to this mail]
Looks good to me!
Quoted 3 lines
Please give it a fast testing. Should we now submit the osdblk driver, for 2.6.31? Which tree? I can push it through the open-osd.org tree. The driver was included for some weeks in linux-next through that tree.
I'll try to give it a test early next week, as my OSD simulator setup is completely disassembled at the moment.
Yes, please go ahead and push the driver. In general, Linus accepts new drivers to the kernel at any time, even during a -rc, once the pre-reqs are upstream.
Jeff
Re: [osd-dev] [PATCH] open-osd: osdblk User Mode utility
In reply to Boaz HarroshQuoted 8 lines · Boaz Harrosh
A minimal user-mode application to Create / Remove / Resize OSD objects from a device, for use with the osdblk.ko block device driver.
See inside patch for Usage instructions.
Signed-off-by Boaz Harrosh <[email protected]>---
This version had a bug which, if the partition did not exist it would fail to create the object.
I have squashed the below patch into this one, on the open-osd git tree
---
Subject: [PATCH] {SQUASHME} open-osd: usr/osdblk: Need to also create the partition
A fall out of the usr/osdblk application
Boaz
--- usr/osdblk.c | 17 +++++++++++++++-- 1 files changed, 15 insertions(+), 2 deletions(-)
usr/osdblk.c +15 -2
@@ -74,7 +74,7 @@ static void usage(void)" pid_no is the partition 64bit number of the object in question\n" " Both 0xabc hex or decimal anotation can be used\n" "\n" -"--obj=obj_no | -o obj_no\n" +"--oid=obj_no | -o obj_no\n" " obj_no is the object 64bit number of the object in question\n" " Both 0xabc hex or decimal anotation can be used\n" "\n"
@@ -198,6 +198,19 @@ static int do_create(struct osd_dev *od, struct osd_obj_id *obj, u64 size)return -ENOMEM ;osdblk_make_credential (creds, obj,osd_req_is_ver1 (or)); + +/* Create partition OK to fail (all ready exist) */ +osd_req_create_partition (or, obj->partition); + ret =osdblk_exec (or, creds); +osd_end_request (or); + +if (ret) +OSDBLK_INFO ("pid=0x%llx exists\n" ,_LLU (obj->partition)); + + or =osd_start_request (od,GFP_KERNEL ); +if (unlikely (!or)) +return -ENOMEM ; +osd_req_create_object (or, obj); ret =osdblk_exec (or, creds);osd_end_request (or);
@@ -280,7 +293,7 @@ int main(int argc, char *argv[]) {.name = "resize", .has_arg = 0, .flag = NULL, .val = 's'} , {.name = "pid", .has_arg = 1, .flag = NULL, .val = 'p'} , {.name = "oid", .has_arg = 1, .flag = NULL, .val = 'o'} , - {.name = "size", .has_arg = 1, .flag = NULL, .val = 'l'} , + {.name = "length", .has_arg = 1, .flag = NULL, .val = 'l'} , {.name = 0, .has_arg = 0, .flag = 0, .val = 0} , };
Signature · 1 line
1.6.2.1