reiser4: batch discard support (FITRIM ioctl): initial implementation.
cover reiser4: batch discard support (FITRIM ioctl): initial implementation.
This patch series implements FITRIM ioctl support in reiser4.
The FITRIM ioctl is supposed to be applied to any directory inside the target filesystem. At least, this can be deduced from fstrim utility behavior on mainstream filesystems (e. g. ext2/3/4).
The idea of implementation is per Edward's advices: FITRIM handler iteratively grabs a portion of disk space, sequentially allocates extents within grabbed space and commits the resulting atom. This way we avoid starving concurrent processes of the disk space. This process is repeated until the partition has been fully scanned and processed.
Points I'm really uncertain of:
- grabbing fixed amount of space (new BA_SOME_SPACE flag, grabs 25% of disk space)
- creation of empty atoms (reiser4_create_atom())
- handling of empty atoms in commit_current_atom()
Ivan Shapovalov (7):
reiser4: block_alloc: add BA_SOME_SPACE flag for grabbing a fixed amount of space.
reiser4: block_alloc: add a "forward" parameter to reiser4_blocknr_hint to allocate blocks only in forward direction.
reiser4: txnmgr: free allocated but unneeded atom in atom_begin_and_assign_to_txnh().
reiser4: txnmgr: add reiser4_create_atom() which creates an empty atom without capturing any nodes.
reiser4: txnmgr: call reiser4_post_write_back_hook() for empty atoms.
reiser4: batch discard support: add a dummy FITRIM ioctl handler for directories.
reiser4: batch discard support: actually implement the FITRIM ioctl handler.
fs/reiser4/block_alloc.c | 6 ++ fs/reiser4/block_alloc.h | 10 ++- fs/reiser4/plugin/dir/dir.h | 2 + fs/reiser4/plugin/file_ops.c | 62 ++++++++++++++++++ fs/reiser4/plugin/object.c | 6 +- fs/reiser4/plugin/space/bitmap.c | 3 +- fs/reiser4/super_ops.c | 131 +++++++++++++++++++++++++++++++++++++++ fs/reiser4/txnmgr.c | 51 ++++++++++----- fs/reiser4/txnmgr.h | 2 + 9 files changed, 252 insertions(+), 21 deletions(-)
Signature · 1 line
2.0.4
RFC PATCH 1/7 reiser4: block_alloc: add BA_SOME_SPACE flag for grabbing a fixed amount of space.
This is used for FITRIM ioctl which will iteratively grab, allocate and trim disk space bit by bit to avoid starving the rest of system.
--- fs/reiser4/block_alloc.c | 6 ++++++ fs/reiser4/block_alloc.h | 5 ++++- 2 files changed, 10 insertions(+), 1 deletion(-)
fs/reiser4/block_alloc.c +6 -0
@@ -276,6 +276,12 @@ reiser4_grab(reiser4_context * ctx, __u64 count, reiser4_ba_flags_t flags) free_blocks = sbinfo->blocks_free; +if (flags &BA_SOME_SPACE ) { +/* Reserve 25% of all free space. */ + count = free_blocks; +do_div (count,4 ); + } +if ((use_reserved && free_blocks < count) || (!use_reserved && free_blocks < count + sbinfo->blocks_reserved)) { ret =RETERR (-ENOSPC );
fs/reiser4/block_alloc.h +4 -1
@@ -79,7 +79,10 @@ enum reiser4_ba_flags {BA_FORCE = (1 <<5 ),/* use default start value for free blocks search. */ - BA_USE_DEFAULT_SEARCH_START = (1 << 6) +BA_USE_DEFAULT_SEARCH_START = (1 <<6 ), + +/* reserve some fixed amount of space */ + BA_SOME_SPACE = (1 << 7), };typedef enum reiser4_ba_flags reiser4_ba_flags_t ;
Signature · 1 line
2.0.4
RFC PATCH 2/7 reiser4: block_alloc: add a "forward" parameter to reiser4_blocknr_hint to allocate blocks only in forward direction.
In reply to Ivan Shapovalov--- fs/reiser4/block_alloc.h | 5 +++-- fs/reiser4/plugin/space/bitmap.c | 3 ++- 2 files changed, 5 insertions(+), 3 deletions(-)
fs/reiser4/block_alloc.h +3 -2
@@ -51,9 +51,10 @@ struct reiser4_blocknr_hint {/* block allocator assumes that blocks, which will be mapped to disk, are in this specified block_stage */ block_stage_t block_stage; -/* If direction = 1 allocate blocks in backward direction from the end -* of disk to the beginning of disk. */ +/* Allocate blocks only in backward direction starting from blk. */ unsigned int backward:1; +/* Allocate blocks only in forward direction starting from blk. */ + unsigned int forward:1; };
fs/reiser4/plugin/space/bitmap.c +2 -1
@@ -1127,7 +1127,8 @@ static int alloc_blocks_forward(reiser4_blocknr_hint *hint, int needed,/* There is only one bitmap search if max_dist was specified or first pass was from the beginning of the bitmap. We also do one pass for scanning bitmap in backward direction. */ -if (!(actual_len !=0 || hint->max_dist !=0 || search_start ==0 )) { +if (actual_len ==0 && search_start !=0 && + hint->max_dist ==0 && hint->forward ==0 ) {/* next step is a scanning from 0 to search_start */ search_end = search_start; search_start =0 ;
Signature · 1 line
2.0.4
RFC PATCH 3/7 reiser4: txnmgr: free allocated but unneeded atom in atom_begin_and_assign_to_txnh().
In reply to Ivan ShapovalovIt is unclear why it hasn't been done before, as there was a corresponding notice, and such change apparently does not carry any side effects.
Do this to avoid duplicating code in the next commit.
--- fs/reiser4/txnmgr.c | 20 +++++--------------- 1 file changed, 5 insertions(+), 15 deletions(-)
fs/reiser4/txnmgr.c +5 -15
@@ -709,6 +709,9 @@ static int atom_begin_and_assign_to_txnh(txn_atom ** atom_alloc, txn_handle * txreturn RETERR (-ENOMEM ); } + atom = *atom_alloc; + *atom_alloc =NULL ; +/* and, also, txnmgr spin lock should be taken before jnode and txnh locks. */ mgr = &get_super_private (reiser4_get_current_sb ())->tmgr;
@@ -717,18 +720,14 @@ static int atom_begin_and_assign_to_txnh(txn_atom ** atom_alloc, txn_handle * tx/* Check whether new atom still needed */ if (txnh->atom !=NULL ) { -/* NOTE-NIKITA probably it is rather better to free -* atom_alloc here than thread it up to reiser4_try_capture() */ -spin_unlock_txnh (txnh);spin_unlock_txnmgr (mgr); +kmem_cache_free (_atom_slab, atom); +return -E_REPEAT ; } - atom = *atom_alloc; - *atom_alloc =NULL ; -atom_init (atom);assert ("jmacd-17" ,atom_isclean (atom));
@@ -2031,15 +2030,6 @@ int reiser4_try_capture(jnode *node, znode_lock_mode lock_mode, goto repeat; } -/* free extra atom object that was possibly allocated by -try_capture_block(). - -Do this before acquiring jnode spin lock to -minimize time spent under lock. --nikita */ -if (atom_alloc !=NULL ) { -kmem_cache_free (_atom_slab, atom_alloc); - } -if (ret !=0 ) {if (ret == -E_BLOCK ) { assert("nikita-3360",
Signature · 1 line
2.0.4
RFC PATCH 4/7 reiser4: txnmgr: add reiser4_create_atom() which creates an empty atom without capturing any nodes.
In reply to Ivan Shapovalov--- fs/reiser4/txnmgr.c | 24 ++++++++++++++++++++++++ fs/reiser4/txnmgr.h | 2 ++ 2 files changed, 26 insertions(+)
fs/reiser4/txnmgr.c +24 -0
@@ -2050,6 +2050,30 @@ int reiser4_try_capture(jnode *node, znode_lock_mode lock_mode,return ret; } +/* This function ensures that the current transcrash has an atom without +* capturing anything. If needed, an empty atom is created and assigned. +*/ +int reiser4_create_atom (void ) +{ +txn_atom *atom_alloc =NULL ; +txn_handle *txnh =get_current_context ()->trans; +int ret; + +do { +spin_lock_txnh (txnh); +if (txnh->atom ==NULL ) { +spin_unlock_txnh (txnh); +/* assign empty atom to the txnh and repeat */ + ret =atom_begin_and_assign_to_txnh (&atom_alloc, txnh); + }else { +spin_unlock_txnh (txnh); + ret =0 ; + } + }while (ret == -E_REPEAT ); + +return ret; +} +static void release_two_atoms (txn_atom *one,txn_atom *two) {spin_unlock_atom (one);
fs/reiser4/txnmgr.h +2 -0
@@ -437,6 +437,8 @@ extern void reiser4_uncapture_jnode(jnode *);extern int reiser4_capture_inode (struct inode *);extern int reiser4_uncapture_inode (struct inode *); +extern int reiser4_create_atom (void ); +extern txn_atom *get_current_atom_locked_nocheck (void );#if REISER4_DEBUG
Signature · 1 line
2.0.4
RFC PATCH 5/7 reiser4: txnmgr: call reiser4_post_write_back_hook() for empty atoms.
In reply to Ivan ShapovalovThe atoms generated by reiser4_trim_fs(), despite being empty (zero capture_count), still have non-empty delete sets which have to be processed.
--- fs/reiser4/txnmgr.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
fs/reiser4/txnmgr.c +6 -1
@@ -1064,8 +1064,13 @@ static int commit_current_atom(long *nr_submitted, txn_atom ** atom)return RETERR (-E_REPEAT ); } -if ((*atom)->capture_count ==0 ) +if ((*atom)->capture_count ==0 ) { +/* Process the atom's delete set. +* Even if the atom has no captured nodes, the delete set may +* still be non-empty (see e. g. reiser4_trim_fs()). */ +reiser4_post_writeback_hook (); goto done; + }/* Up to this point we have been flushing and after flush is called we return -E_REPEAT. Now we can commit. We cannot return -E_REPEAT
Signature · 1 line
2.0.4
Re: [RFC] [PATCH 5/7] reiser4: txnmgr: call reiser4_post_write_back_hook() for empty atoms.
Quoted 14 lines · Ivan Shapovalov
The atoms generated by
reiser4_trim_fs(), despite being empty (zerocapture_count), still have non-empty delete sets which have to be processed.Signed-off-by Ivan Shapovalov <[email protected]>--- fs/reiser4/txnmgr.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)fs/reiser4/txnmgr.c +0
-0
fs/reiser4/txnmgr.c static int commit_current_atom(long *nr_submitted, txn_atom ** atom) · line 1071
Quoted 9 lines
fs/reiser4/txnmgr.c +5
-1@@ -1064,8 +1064,13 @@ static int commit_current_atom(long *nr_submitted, txn_atom ** atom)return RETERR (-E_REPEAT ); } -if ((*atom)->capture_count ==0 ) +if ((*atom)->capture_count ==0 ) { +/* Process the atom's delete set. +* Even if the atom has no captured nodes, the delete set may +* still be non-empty (see e. g. reiser4_trim_fs()). */ +reiser4_post_writeback_hook ();
Oops, a typo. Will resend the series...
Signature · 1 line
Ivan Shapovalov / intelfx /
Quoted 6 lines
fs/reiser4/txnmgr.c +1
-0goto done; + }/* Up to this point we have been flushing and after flush is called we return -E_REPEAT. Now we can commit. We cannot return -E_REPEAT
Re: [RFC] [PATCH 5/7] reiser4: txnmgr: call reiser4_post_write_back_hook() for empty atoms.
Quoted 18 lines · Ivan Shapovalov
Quoted 17 lines · Ivan Shapovalov
The atoms generated by
reiser4_trim_fs(), despite being empty (zerocapture_count), still have non-empty delete sets which have to be processed.Signed-off-by Ivan Shapovalov <[email protected]>--- fs/reiser4/txnmgr.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)fs/reiser4/txnmgr.c +0
-0@@ -1064,8 +1064,13 @@ static int commit_current_atom(long *nr_submitted, txn_atom ** atom)return RETERR (-E_REPEAT ); }
Quoted 8 lines
fs/reiser4/txnmgr.c static int commit_current_atom(long *nr_submitted, txn_atom ** atom) · line 1071
Quoted 6 lines
fs/reiser4/txnmgr.c +5
-1-if ((*atom)->capture_count ==0 ) +if ((*atom)->capture_count ==0 ) { +/* Process the atom's delete set. +* Even if the atom has no captured nodes, the delete set may +* still be non-empty (see e. g. reiser4_trim_fs()). */ +reiser4_post_writeback_hook ();Oops, a typo. Will resend the series...
Actually, it's not just a typo. reiser4_post_write_back_hook() shall be called with atom spinlock released. See PATCHv2, and sorry for the noise.
Signature · 1 line
Ivan Shapovalov / intelfx /
Quoted 7 lines
Quoted 7 lines
fs/reiser4/txnmgr.c +1
-0goto done; + }/* Up to this point we have been flushing and after flush is called we return -E_REPEAT. Now we can commit. We cannot return -E_REPEAT
RFC PATCH 6/7 reiser4: batch discard support: add a dummy FITRIM ioctl handler for directories.
In reply to Ivan Shapovalov--- fs/reiser4/plugin/dir/dir.h | 2 ++ fs/reiser4/plugin/file_ops.c | 27 +++++++++++++++++++++++++++ fs/reiser4/plugin/object.c | 6 +++++- 3 files changed, 34 insertions(+), 1 deletion(-)
fs/reiser4/plugin/dir/dir.h +2 -0
@@ -10,6 +10,8 @@#include <linux/fs.h>*/ +long reiser4_ioctl_dir_common (struct file *file,unsigned int cmd,unsigned long arg); +/* declarations of functions implementing HASHED_DIR_PLUGIN_ID dir plugin */ /* "hashed" directory methods of dir plugin */
fs/reiser4/plugin/file_ops.c +27 -0
@@ -107,6 +107,33 @@ int reiser4_sync_file_common(struct file *file, loff_t start, loff_t end, int dareturn 0 ; } +/** reiser4_ioctl_dir_common - ioctl of struct file_operations for typical directory +*/ +long reiser4_ioctl_dir_common (struct file *file,unsigned int cmd,unsigned long arg) +{ +struct inode *inode =file_inode (file); +struct super_block *super = inode->i_sb; +reiser4_context *ctx; +int ret; + + ctx =reiser4_init_context (super); +if (IS_ERR (ctx)) +return PTR_ERR (ctx); + +switch (cmd) { +case FITRIM : +warning ("intelfx-62" ,"FITRIM ioctl not implemented" ); +/* fall-through to -ENOSYS */ + +default : + ret =RETERR (-ENOSYS ); +break ; + } + +reiser4_exit_context (ctx); +return ret; +} +/* * Local variables: * c-indentation-style: "K&R"
fs/reiser4/plugin/object.c +5 -1
@@ -251,7 +251,11 @@ static struct file_operations directory_f_ops = { .read = generic_read_dir, .iterate = reiser4_iterate_common, .release = reiser4_release_dir_common, - .fsync = reiser4_sync_common + .fsync = reiser4_sync_common, +#ifdef CONFIG_COMPAT + .compat_ioctl = reiser4_ioctl_dir_common, +#endif + .unlocked_ioctl = reiser4_ioctl_dir_common };static struct address_space_operations directory_a_ops = { .writepage = writepage_bugop,
Signature · 1 line
2.0.4
RFC PATCH 7/7 reiser4: batch discard support: actually implement the FITRIM ioctl handler.
In reply to Ivan ShapovalovIt works in an iterative way, grabbing some fixed amount of space and allocating blocks until grabbed space is exhausted or partition's end is reached.
Blocks are scheduled for discard by allocating them and immediately adding them to the atom's delete set, which is read and processed at atom commit time.
After reaching the allocation stop condition, the atom is force-committed and, if the allocation has been stopped due to grabbed space exhaustion (not due to reaching end of partition), another iteration happens.
--- fs/reiser4/plugin/file_ops.c | 41 +++++++++++++- fs/reiser4/super_ops.c | 131 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 169 insertions(+), 3 deletions(-)
fs/reiser4/plugin/file_ops.c +38 -3
@@ -8,6 +8,9 @@#include "../inode.h" #include "object.h" +#include <linux/fs.h> +#include <linux/blkdev.h> +/* file operations */ /* implementation of vfs's llseek method of struct file_operations for
@@ -20,6 +23,9 @@ loff_t reiser4_llseek_dir_common(struct file *, loff_t, int origin);*/ int reiser4_iterate_common (struct file *,struct dir_context *); +/* this function is implemented in super_ops.c */ +int reiser4_trim_fs (struct super_block *super,struct fstrim_range * range); +/** * reiser4_release_dir_common - release of struct file_operations * @inode: inode of released file
@@ -121,9 +127,38 @@ long reiser4_ioctl_dir_common(struct file *file, unsigned int cmd, unsigned longreturn PTR_ERR (ctx);switch (cmd) { -case FITRIM : -warning ("intelfx-62" ,"FITRIM ioctl not implemented" ); -/* fall-through to -ENOSYS */ +case FITRIM : { +struct request_queue *q =bdev_get_queue (super->s_bdev); +struct fstrim_range range; + +if (!capable (CAP_SYS_ADMIN )) { + ret =RETERR (-EPERM ); +break ; + } + +if (!blk_queue_discard (q)) { + ret =RETERR (-EOPNOTSUPP ); +break ; + } + +if (copy_from_user (&range, (struct fstrim_range __user *)arg, +sizeof (range))) { + ret =RETERR (-EFAULT ); +break ; + } + + range.minlen =max ((unsigned int )range.minlen, + q->limits.discard_granularity); + + ret =reiser4_trim_fs (super, &range); + +if (copy_to_user ((struct fstrim_range __user *)arg, &range, +sizeof (range))) { + ret =RETERR (-EFAULT ); + } + +break ; + }default : ret =RETERR (-ENOSYS );
fs/reiser4/super_ops.c +131 -0
143 lines not on this page · raw message
Signature · 1 line
2.0.4
A message that is not in the archive (sent off-list or not yet received): CADW=+3kCURR6UB5Tm8OcJnSYV1cNcYtTmbNNrv_QCO3o1RzwDQ@mail.gmail.com
Quoted 7 lines · Dušan Čolić
[...]
Quoted 3 lines
Points I'm really uncertain of:
- grabbing fixed amount of space (newBA_SOME_SPACEflag, grabs 25% ofdisk space)
Maybe grabbing 25% of remaining empty disk space would be better?
What do you mean by "remaining"? 25% is recalculated on each iteration. However, the reserved and allocated disk space is reclaimed at the end of each iteration, so the amount really stays the same.
Signature · 1 line
Ivan Shapovalov / intelfx /
Quoted 5 lines
Quoted 4 lines
- creation of empty atoms (
reiser4_create_atom())
- handling of empty atoms incommit_current_atom()[...]
A message that is not in the archive (sent off-list or not yet received): CADW=+3=B_3pKNzph7EtfXNLbVxxn4dxsykm_RF+ayce9RsoVUQ@mail.gmail.com
Quoted 11 lines · Dušan Čolić
Quoted 10 lines · Ivan Shapovalov
Quoted 7 lines · Dušan Čolić
[...]
Quoted 3 lines
Points I'm really uncertain of:
- grabbing fixed amount of space (newBA_SOME_SPACEflag, grabs 25% ofdisk space)
Maybe grabbing 25% of remaining empty disk space would be better?
Quoted 8 lines
Quoted 4 lines
What do you mean by "remaining"? 25% is recalculated on each iteration. However, the reserved and allocated disk space is reclaimed at the end of each iteration, so the amount really stays the same.
I meant not to grab 25% of whole partition size but grab amount of disk space equal to 25% of empty partition space so if we have only 10% of free space left we grab only 2.5% of it per iteration so to not get ENOSPACE if we used 25% of whole disk size and grabbed whole remaining disk space.
Ah. I've meant "25% of free disk space at the moment of discarding". It's already done the correct way, I've just misworded it.
Signature · 1 line
Ivan Shapovalov / intelfx /
Quoted 9 lines
Quoted 9 lines
--
Ivan Shapovalov / intelfx /Quoted 5 lines
Quoted 4 lines
- creation of empty atoms (
reiser4_create_atom())
- handling of empty atoms incommit_current_atom()[...]