lustre: merged OpenSFS client patches from April 30 to today
cover lustre: merged OpenSFS client patches from April 30 to today
Merge the client side patches that landed to the OpenSFS tree since Apirl 30 to today. Include a fid missing patches as well and one patch to fix issues with 2 patches from the earlier 600+ patch set. Please review to ensure their correctness.
Alexander Boyko (1):
lustre: llog: allow delete of zero size llog
Amir Shehata (2):
lnet: restrict gateway selection
lnet: use the same src nid for discovery
Andreas Dilger (5):
lustre: llite: verify truncated xattr is handled
lustre: obd: fix printing of client connection UUID
lustre: llite: restore ll_dcompare()
lustre: ldlm: use proper units for timeouts
lustre: lmv: do not print MDTs that are inactive
Andriy Skulysh (1):
lustre: ptlrpc: connect to MDT stucks
Bobi Jam (2):
lustre: lov: lov_io_sub_init()) ASSERTION
lustre: llite: do not flush COW pages from mapping
Chris Horn (2):
lnet: Introduce constant for the lolnd NID
lnet: Add MD options for response tracking
James Simmons (2):
lustre: cleanups and bug fixes
lnet: stop using struct timeval
Lai Siyao (3):
lustre: uapi: add OBD_CONNECT2_FIDMAP
lustre: dne: support directory restripe
lustre: dne: improve temp file name check
Mr NeilBrown (13):
lnet: merge lnet_md_alloc into lnet_md_build.
lnet: always put a page list into struct lnet_libmd
lnet: discard kvec option from lnet_libmd.
lnet: remove msg_iov from lnet_msg.
lnet: o2iblnd: discard kiblnd_setup_rd_iov
lustre: Remove inappropriate uses of BIT() macro.
lnet: libcfs: use BIT() macro where appropriate
lustre: llite: clean up pcc_layout_wait()
lustre: misc: declare static chars as const where possible.
lnet: use kmem_cache_zalloc as appropriate.
lnet: remove lnet_extract_iov()
lnet: simplify ksock_tx.
lnet: socklnd: discard tx_iov.
NeilBrown (2):
lustre: fid: revert seq_client_rpc patch.
lustre: fld: convert cache_flush file to LPROC_SEQ_FOPS
Oleg Drokin (3):
lustre: ptlrpc: return proper write count from ping_store
lustre: osc: Do not wait for grants for too long
lustre: osc: Ensure immediate departure of sync write pages
Qian Yingjin (1):
lustre: Send file creation time to clients
Sebastien Buisson (2):
lustre: sec: check permissions for changelogs access
lustre: mgc: protect from NULL exp in mgc_enqueue()
Sergey Cheremencev (1):
lustre: quota: quota pools for OSTs
Shaun Tancheff (1):
lustre: all: Cleanup LASSERTF uses missing newlines
Swapnil Pimpale (1):
lustre: fallocate: Implement fallocate preallocate operation
Wang Shilong (3):
lustre: llite: fix to make jobstats work for async ra
lustre: llite: fix possible divide zero in ll_use_fast_io()
lustre: llite: check if page truncated in ll_write_begin()
fs/lustre/fid/fid_request.c | 12 +-- fs/lustre/fld/fld_cache.c | 40 -------- fs/lustre/fld/fld_internal.h | 7 -- fs/lustre/fld/lproc_fld.c | 31 +++--- fs/lustre/include/cl_object.h | 44 ++++++++- fs/lustre/include/lu_object.h | 16 +-- fs/lustre/include/lustre_dlm.h | 17 ++-- fs/lustre/include/lustre_import.h | 4 +- fs/lustre/include/lustre_lmv.h | 94 ++++++++++++++++-- fs/lustre/include/lustre_req_layout.h | 3 + fs/lustre/include/lustre_swab.h | 2 +- fs/lustre/include/obd_support.h | 5 +- fs/lustre/include/obd_target.h | 54 ++++++++++ fs/lustre/ldlm/ldlm_lockd.c | 3 +- fs/lustre/ldlm/ldlm_request.c | 1 + fs/lustre/ldlm/ldlm_resource.c | 11 ++- fs/lustre/llite/dcache.c | 56 ++++++++--- fs/lustre/llite/dir.c | 33 +++++-- fs/lustre/llite/file.c | 84 +++++++++++++++- fs/lustre/llite/lcommon_cl.c | 2 + fs/lustre/llite/llite_internal.h | 6 +- fs/lustre/llite/llite_lib.c | 6 +- fs/lustre/llite/llite_mmap.c | 2 +- fs/lustre/llite/lproc_llite.c | 19 ++-- fs/lustre/llite/namei.c | 5 +- fs/lustre/llite/pcc.c | 17 ++-- fs/lustre/llite/rw.c | 13 ++- fs/lustre/llite/rw26.c | 16 ++- fs/lustre/llite/statahead.c | 2 +- fs/lustre/llite/vvp_io.c | 19 ++-- fs/lustre/llite/vvp_object.c | 2 +- fs/lustre/llite/xattr.c | 15 ++- fs/lustre/lmv/lmv_obd.c | 66 +++++++++---- fs/lustre/lov/lov_cl_internal.h | 6 +- fs/lustre/lov/lov_io.c | 31 ++++-- fs/lustre/mdc/mdc_changelog.c | 1 + fs/lustre/mdc/mdc_request.c | 41 +++++--- fs/lustre/mgc/mgc_request.c | 11 ++- fs/lustre/obdclass/cl_page.c | 1 - fs/lustre/obdclass/class_obd.c | 35 +++---- fs/lustre/obdclass/llog.c | 9 +- fs/lustre/obdclass/llog_cat.c | 3 +- fs/lustre/obdclass/lprocfs_status.c | 8 +- fs/lustre/obdclass/obd_mount.c | 2 +- fs/lustre/osc/osc_cache.c | 33 ++++++- fs/lustre/osc/osc_internal.h | 3 + fs/lustre/osc/osc_io.c | 56 +++++++++-- fs/lustre/osc/osc_request.c | 70 ++++++++++++- fs/lustre/ptlrpc/client.c | 2 +- fs/lustre/ptlrpc/events.c | 2 +- fs/lustre/ptlrpc/import.c | 4 +- fs/lustre/ptlrpc/layout.c | 26 ++++- fs/lustre/ptlrpc/llog_client.c | 4 + fs/lustre/ptlrpc/lproc_ptlrpc.c | 4 +- fs/lustre/ptlrpc/pack_generic.c | 16 ++- fs/lustre/ptlrpc/ptlrpc_internal.h | 19 ++++ fs/lustre/ptlrpc/wiretest.c | 16 ++- include/linux/lnet/lib-lnet.h | 56 +---------- include/linux/lnet/lib-types.h | 15 +-- include/uapi/linux/lnet/lnet-types.h | 11 +++ include/uapi/linux/lnet/lnetst.h | 8 +- include/uapi/linux/lustre/lustre_idl.h | 28 ++++-- include/uapi/linux/lustre/lustre_user.h | 51 +++++++++- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 122 +++-------------------- net/lnet/klnds/socklnd/socklnd.h | 16 +-- net/lnet/klnds/socklnd/socklnd_cb.c | 68 +++++-------- net/lnet/klnds/socklnd/socklnd_lib.c | 32 +++--- net/lnet/klnds/socklnd/socklnd_proto.c | 10 +- net/lnet/libcfs/debug.c | 2 +- net/lnet/libcfs/libcfs_string.c | 4 +- net/lnet/libcfs/module.c | 17 ++-- net/lnet/lnet/api-ni.c | 16 +-- net/lnet/lnet/config.c | 16 +-- net/lnet/lnet/lib-md.c | 168 ++++++++++++++++---------------- net/lnet/lnet/lib-me.c | 2 +- net/lnet/lnet/lib-move.c | 167 ++++++++++++++----------------- net/lnet/lnet/lib-msg.c | 5 +- net/lnet/lnet/lib-socket.c | 34 ++----- net/lnet/lnet/lo.c | 17 +--- net/lnet/lnet/peer.c | 17 +++- net/lnet/lnet/router.c | 2 +- net/lnet/selftest/conctl.c | 1 - net/lnet/selftest/conrpc.c | 6 +- net/lnet/selftest/conrpc.h | 1 - net/lnet/selftest/console.h | 1 - 85 files changed, 1201 insertions(+), 802 deletions(-) create mode 100644 fs/lustre/include/obd_target.h
Signature · 1 line
1.8.3.1
PATCH 1/45 lustre: fid: revert seq_client_rpc patch.
From: NeilBrown <[email protected]>
This reverts commit 5955572b19f6 ("staging: lustre: fid: perform sanity checks before committing"). The patch isn't really needed, and keeping it caused unnecessary differences from out-of-free lustre.
--- fs/lustre/fid/fid_request.c | 10 +++++----- fs/lustre/include/lustre_lmv.h | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-)
fs/lustre/fid/fid_request.c +5 -5
@@ -118,22 +118,22 @@ static int seq_client_rpc(struct lu_client_seq *seq, goto out_req; out =req_capsule_server_get (&req->rq_pill, &RMF_SEQ_RANGE ); + *output = *out; -if (!lu_seq_range_is_sane (out)) { +if (!lu_seq_range_is_sane (output)) {CERROR ("%s: Invalid range received from server: " -DRANGE "\n" , seq->lcs_name,PRANGE (out)); +DRANGE "\n" , seq->lcs_name,PRANGE (output)); rc = -EINVAL ; goto out_req; } -if (lu_seq_range_is_exhausted (out)) { +if (lu_seq_range_is_exhausted (output)) {CERROR ("%s: Range received from server is exhausted: " -DRANGE "]\n" , seq->lcs_name,PRANGE (out)); +DRANGE "]\n" , seq->lcs_name,PRANGE (output)); rc = -EINVAL ; goto out_req; } - *output = *out;CDEBUG_LIMIT (debug_mask,"%s: Allocated %s-sequence " DRANGE "]\n" , seq->lcs_name, opcname,PRANGE (output));
fs/lustre/include/lustre_lmv.h +1 -1
@@ -68,7 +68,7 @@ static inline bool lmv_dir_foreign(const struct lmv_stripe_md *lsm)static inline bool lmv_dir_layout_changing (const struct lmv_stripe_md *lsm) {return lmv_dir_striped (lsm) && - (lsm->lsm_md_hash_type &LMV_HASH_FLAG_MIGRATION ); + (lsm->lsm_md_hash_type &LMV_HASH_FLAG_LAYOUT_CHANGE ); }static inline bool lmv_dir_bad_hash (const struct lmv_stripe_md *lsm)
Signature · 1 line
1.8.3.1
PATCH 2/45 lustre: fld: convert cache_flush file to LPROC_SEQ_FOPS
In reply to James SimmonsFrom: NeilBrown <[email protected]>
Commit 827650494fbe ("staging/lustre/fld: move all files from procfs to debugfs") converted files to debugfs, but missed the opportunity to provide a simpler implementation for cache_flush. When that patch was taken to OpenSFS lustre in Commit 6d2b66d22c25 ("LU-8066 fld: move all files from procfs to debugfs"), that opportunity was taken.
So we now copy that improvement back to Linux.
--- fs/lustre/fld/lproc_fld.c | 31 +++++++++++-------------------- fs/lustre/include/lustre_lmv.h | 4 ++-- fs/lustre/llite/dcache.c | 3 +-- fs/lustre/llite/llite_internal.h | 4 ++-- fs/lustre/llite/lproc_llite.c | 4 ++-- fs/lustre/llite/namei.c | 2 +- 6 files changed, 19 insertions(+), 29 deletions(-)
fs/lustre/fld/lproc_fld.c +11 -20
@@ -117,10 +117,11 @@ }static ssize_t -fld_debugfs_cache_flush_write (struct file *file,const char __user *buffer, -size_t count,loff_t *pos) +lprocfs_wr_cache_flush (struct file *file,const char __user *buffer, +size_t count,loff_t *pos) { -struct lu_client_fld *fld = file->private_data; +struct seq_file *m = file->private_data; +struct lu_client_fld *fld = m->private;fld_cache_flush (fld->lcf_cache);
@@ -129,26 +130,16 @@return count; } -static int -fld_debugfs_cache_flush_release (struct inode *inode,struct file *file) -{ - file->private_data =NULL ; -return 0 ; -} - -static const struct file_operations fld_debugfs_cache_flush_fops = { - .owner =THIS_MODULE , - .open = simple_open, - .write = fld_debugfs_cache_flush_write, - .release = fld_debugfs_cache_flush_release, -}; -LPROC_SEQ_FOPS_RO (fld_debugfs_targets);LPROC_SEQ_FOPS (fld_debugfs_hash); +LPROC_SEQ_FOPS_WR_ONLY (fld, cache_flush);struct lprocfs_vars fld_client_debugfs_list[] = { - {"targets" , &fld_debugfs_targets_fops }, - {"hash" , &fld_debugfs_hash_fops }, - {"cache_flush" , &fld_debugfs_cache_flush_fops }, + { .name ="targets" , + .fops = &fld_debugfs_targets_fops }, + { .name ="hash" , + .fops = &fld_debugfs_hash_fops }, + { .name ="cache_flush" , + .fops = &fld_cache_flush_fops }, {NULL } };
fs/lustre/include/lustre_lmv.h +2 -2
@@ -299,8 +299,8 @@ static inline u32 crush_hash(u32 a, u32 b)if (stripe_count >1 ) {switch (hash_type &LMV_HASH_TYPE_MASK ) {case LMV_HASH_TYPE_ALL_CHARS : - stripe_index =lmv_hash_all_chars (stripe_count, name, - namelen); + stripe_index =lmv_hash_all_chars (stripe_count, name, + namelen);break ;case LMV_HASH_TYPE_FNV_1A_64 : stripe_index = lmv_hash_fnv1a(stripe_count, name,
fs/lustre/llite/dcache.c +1 -2
@@ -58,7 +58,6 @@ static void ll_release(struct dentry *de)LASSERT (de); lld =ll_d2d (de); -call_rcu (&lld->lld_rcu_head, free_dentry_data); }
@@ -118,7 +117,7 @@ void ll_intent_drop_lock(struct lookup_intent *it) handle.cookie = it->it_remote_lock_handle;CDEBUG (D_DLMTRACE , -"releasing remote lock with cookie%#llx from it %p\n" , +"releasing remote lock with cookie %#llx from it %p\n" , handle.cookie, it);ldlm_lock_decref (&handle, it->it_remote_lock_mode);
fs/lustre/llite/llite_internal.h +2 -2
@@ -414,10 +414,10 @@ static inline struct pcc_inode *ll_i2pcci(struct inode *inode)/* default to use at least 16M for fast read if possible */ #define RA_REMAIN_WINDOW_MIN MiB_TO_PAGES(16UL) -/* default to about 64M of readahead on a given system. */ +/* default readahead on a given system. */ #define SBI_DEFAULT_READ_AHEAD_MAX MiB_TO_PAGES(64UL) -/* default to read-ahead full files smaller than 2MB on the second read */ +/* default read-ahead full files smaller than 2MB on the second read */ #define SBI_DEFAULT_READ_AHEAD_WHOLE_MAX MiB_TO_PAGES(2UL)enum ra_stat {
fs/lustre/llite/lproc_llite.c +2 -2
@@ -354,7 +354,7 @@ static ssize_t max_read_ahead_mb_store(struct kobject *kobj, pages_number =round_up (ra_max_mb,1024 *1024 ) >>PAGE_SHIFT ;if (pages_number >totalram_pages () /2 ) {/* 1/2 of RAM */ -CERROR ("%s: cannot set max_readahead_mb=%llu > %luMB\n" , +CERROR ("%s: cannot set max_read_ahead_mb=%llu > totalram/2=%luMB\n" , sbi->ll_fsname,PAGES_TO_MiB (pages_number),PAGES_TO_MiB (totalram_pages () /2 ));return -ERANGE ;
@@ -399,7 +399,7 @@ static ssize_t max_read_ahead_per_file_mb_store(struct kobject *kobj, pages_number =round_up (ra_max_file_mb,1024 *1024 ) >>PAGE_SHIFT ;if (pages_number > sbi->ll_ra_info.ra_max_pages) { -CERROR ("%s: cannot set max_readahead_per_file_mb=%llu > max_read_ahead_mb=%lu\n" , +CERROR ("%s: cannot set max_read_ahead_per_file_mb=%llu > max_read_ahead_mb=%lu\n" , sbi->ll_fsname,PAGES_TO_MiB (pages_number),PAGES_TO_MiB (sbi->ll_ra_info.ra_max_pages));return -ERANGE ;
fs/lustre/llite/namei.c +1 -1
@@ -978,7 +978,7 @@ static int ll_atomic_open(struct inode *dir, struct dentry *dentry,int rc =0 ;CDEBUG (D_VFSTRACE , -"VFS Op:name=%pd, dir=" DFID "(%p),file %p,open_flags %x,mode %x\n" , +"VFS Op:name=%pd, dir=" DFID "(%p), file %p, open_flags %x, mode %x\n" , dentry,PFID (ll_inode2fid (dir)), dir, file, open_flags, mode);/* Only negative dentries enter here */
Signature · 1 line
1.8.3.1
PATCH 3/45 lustre: cleanups and bug fixes
In reply to James SimmonsFix various bugs in the Linux Lustre client. Small styles fixes as well.
--- fs/lustre/fid/fid_request.c | 2 +- fs/lustre/fld/fld_cache.c | 40 ------------------------------------- fs/lustre/fld/fld_internal.h | 7 ------- fs/lustre/llite/llite_internal.h | 2 +- fs/lustre/llite/llite_lib.c | 2 -- fs/lustre/llite/lproc_llite.c | 10 +++++----- fs/lustre/llite/statahead.c | 2 +- fs/lustre/llite/xattr.c | 1 + fs/lustre/mdc/mdc_changelog.c | 1 + fs/lustre/mgc/mgc_request.c | 8 +++++--- fs/lustre/obdclass/cl_page.c | 1 - fs/lustre/obdclass/class_obd.c | 23 ++++++++++----------- fs/lustre/obdclass/lprocfs_status.c | 6 +++--- fs/lustre/osc/osc_cache.c | 2 +- fs/lustre/osc/osc_request.c | 5 ++--- fs/lustre/ptlrpc/import.c | 2 +- 16 files changed, 33 insertions(+), 81 deletions(-)
fs/lustre/fid/fid_request.c +1 -1
@@ -179,7 +179,7 @@ static int seq_client_alloc_seq(const struct lu_env *env, rc =seq_client_alloc_meta (env, seq);if (rc) {if (rc != -EINPROGRESS ) -CERROR ("%s: Can't allocate new meta-sequence, rc = %d\n" , +CERROR ("%s: Cannot allocate new meta-sequence, rc = %d\n" , seq->lcs_name, rc); *seqnr =U64_MAX ;return rc;
fs/lustre/fld/fld_cache.c +0 -40
@@ -431,46 +431,6 @@ int fld_cache_insert(struct fld_cache *cache, }/** -* Delete FLD entry in FLD cache. -* -*/ - -struct fld_cache_entry -*fld_cache_entry_lookup_nolock (struct fld_cache *cache, -struct lu_seq_range *range) -{ -struct fld_cache_entry *flde; -struct fld_cache_entry *got =NULL ; -struct list_head *head; - - head = &cache->fci_entries_head; -list_for_each_entry (flde, head, fce_list) { -if (range->lsr_start == flde->fce_range.lsr_start || - (range->lsr_end == flde->fce_range.lsr_end && - range->lsr_flags == flde->fce_range.lsr_flags)) { - got = flde; -break ; - } - } - -return got; -} - -/** -* lookup @seq sequence for range in fld cache. -*/ -struct fld_cache_entry -*fld_cache_entry_lookup (struct fld_cache *cache,struct lu_seq_range *range) -{ -struct fld_cache_entry *got =NULL ; - -read_lock (&cache->fci_lock); - got =fld_cache_entry_lookup_nolock (cache, range); -read_unlock (&cache->fci_lock); -return got; -} - -/** * lookup @seq sequence for range in fld cache. */ int fld_cache_lookup(struct fld_cache *cache,
fs/lustre/fld/fld_internal.h +0 -7
@@ -151,13 +151,6 @@ struct fld_cache_entryint fld_cache_lookup (struct fld_cache *cache,const u64 seq,struct lu_seq_range *range); -struct fld_cache_entry * -fld_cache_entry_lookup (struct fld_cache *cache,struct lu_seq_range *range); - -struct fld_cache_entry -*fld_cache_entry_lookup_nolock (struct fld_cache *cache, -struct lu_seq_range *range); -static inline const char *fld_target_name (struct lu_fld_target *tar) {
fs/lustre/llite/llite_internal.h +1 -1
@@ -417,7 +417,7 @@ static inline struct pcc_inode *ll_i2pcci(struct inode *inode)/* default readahead on a given system. */ #define SBI_DEFAULT_READ_AHEAD_MAX MiB_TO_PAGES(64UL) -/* default read-ahead full files smaller than 2MB on the second read */ +/* default read-ahead full files smaller than limit on the second read */ #define SBI_DEFAULT_READ_AHEAD_WHOLE_MAX MiB_TO_PAGES(2UL)enum ra_stat {
fs/lustre/llite/llite_lib.c +0 -2
@@ -2347,8 +2347,6 @@ void ll_umount_begin(struct super_block *sb)ssleep (1 ); cnt++; } - -schedule (); }int ll_remount_fs (struct super_block *sb,int *flags,char *data)
fs/lustre/llite/lproc_llite.c +5 -5
@@ -349,9 +349,9 @@ static ssize_t max_read_ahead_mb_store(struct kobject *kobj,if (rc)return rc; + pages_number =round_up (ra_max_mb,1024 *1024 ) >>PAGE_SHIFT ;CDEBUG (D_INFO ,"%s: set max_read_ahead_mb=%llu (%llu pages)\n" , sbi->ll_fsname,PAGES_TO_MiB (pages_number), pages_number); - pages_number =round_up (ra_max_mb,1024 *1024 ) >>PAGE_SHIFT ;if (pages_number >totalram_pages () /2 ) {/* 1/2 of RAM */ CERROR("%s: cannot set max_read_ahead_mb=%llu > totalram/2=%luMB\n",
@@ -571,7 +571,7 @@ static ssize_t ll_max_cached_mb_seq_write(struct file *file,break ;if (!sbi->ll_dt_exp) {/* being initialized */ - rc =0 ; + rc = -ENODEV ; goto out; }
@@ -1111,9 +1111,9 @@ static ssize_t max_read_ahead_async_active_show(struct kobject *kobj, }static ssize_t max_read_ahead_async_active_store (struct kobject *kobj, -struct attribute *attr, -const char *buffer, -size_t count) +struct attribute *attr, +const char *buffer, +size_t count) { struct ll_sb_info *sbi = container_of(kobj, struct ll_sb_info, ll_kset.kobj);
fs/lustre/llite/statahead.c +1 -1
@@ -1568,7 +1568,7 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry)* statahead thread won't be notified to quit. */ spin_lock (&lli->lli_sa_lock); -if (unlikely (lli->lli_sai || lli->lli_opendir_key || +if (unlikely (lli->lli_sai || !lli->lli_opendir_key || lli->lli_opendir_pid != current->pid)) {spin_unlock (&lli->lli_sa_lock); rc = -EPERM ;
fs/lustre/llite/xattr.c +1 -0
@@ -264,6 +264,7 @@ static int ll_setstripe_ea(struct dentry *dentry, struct lov_user_md *lump,*/ return -ERANGE ; } + rc =ll_adjust_lum (inode, lump);if (rc)return rc;
fs/lustre/mdc/mdc_changelog.c +1 -0
@@ -840,6 +840,7 @@ int mdc_changelog_cdev_init(struct obd_device *obd) entry->ced_device = device; entry =NULL ;/* prevent it from being freed below */ + rc =0 ; goto out_unlock; out_cdev:
fs/lustre/mgc/mgc_request.c +5 -3
@@ -970,9 +970,11 @@ static int mgc_set_info_async(const struct lu_env *env, struct obd_export *exp, imp->imp_deactive, imp->imp_invalid, imp->imp_replayable, imp->imp_obd->obd_replayable, ptlrpc_import_state_name(imp->imp_state)); -/* Resurrect if we previously died */ -if ((imp->imp_state !=LUSTRE_IMP_FULL && - imp->imp_state !=LUSTRE_IMP_NEW ) || value >1 ) +/* Resurrect the import immediately if +* 1. we previously got disconnected, +* 2. value > 1 (at the same node with MGS) +*/ +if (imp->imp_state !=LUSTRE_IMP_NEW || value >1 )ptlrpc_reconnect_import (imp);return 0 ; }
fs/lustre/obdclass/cl_page.c +0 -1
@@ -200,7 +200,6 @@ struct cl_page *cl_page_alloc(const struct lu_env *env,struct cl_page *page;struct cl_object *o2; - page =kzalloc (cl_object_header (o)->coh_page_bufsize,GFP_NOFS ); page =__cl_page_alloc (o);if (page) {int result =0 ;
fs/lustre/obdclass/class_obd.c +11 -12
@@ -512,7 +512,7 @@ static long obd_class_ioctl(struct file *filp, unsigned int cmd,u64 __size; \int __ret; \ \ -BUILD_BUG_ON (strlen (value) >=23 ); \ +BUILD_BUG_ON (sizeof (value) >=23 ); \ __ret =sysfs_memparse (value,sizeof (value) -1 , &__size, def_unit); \if (__ret != __rc) \ CERROR("string_helper: parsing '%s' expect rc %d != got %d\n", \
@@ -531,9 +531,6 @@ static int __init obd_init_checks(void)char buf[64 ];int len, ret =0 ; -CDEBUG (D_INFO ,"LPU64=%s, LPD64=%s, LPX64=%s\n" ,"%llu" ,"%lld" , -"%#llx" ); -CDEBUG (D_INFO ,"OBD_OBJECT_EOF = %#llx\n" , (u64 )OBD_OBJECT_EOF ); u64val =OBD_OBJECT_EOF ;
@@ -545,7 +542,7 @@ static int __init obd_init_checks(void) } len =snprintf (buf,sizeof (buf),"%#llx" , u64val);if (len !=18 ) { -CWARN ("LPX64 wrong length! strlen(%s)=%d != 18\n" , buf, len); +CERROR ("LPX64 wrong length! strlen(%s)=%d != 18\n" , buf, len); ret = -EINVAL ; }
@@ -559,37 +556,39 @@ static int __init obd_init_checks(void)if (u64val >>8 !=OBD_OBJECT_EOF >>8 ) {CERROR ("u64 %#llx(%d) != 0xffffffffffffffff\n" , u64val, (int )sizeof (u64val)); -return -EOVERFLOW ; + ret = -EOVERFLOW ; }if (do_div (div64val,256 ) != (u64val &255 )) {CERROR ("do_div(%#llx,256) != %llu\n" , u64val, u64val &255 ); -return -EOVERFLOW ; + ret = -EOVERFLOW ; }if (u64val >>8 != div64val) {CERROR ("do_div(%#llx,256) %llu != %llu\n" , u64val, div64val, u64val >>8 ); -return -EOVERFLOW ; + ret = -EOVERFLOW ; } len =snprintf (buf,sizeof (buf),"%#llx" , u64val);if (len !=18 ) { -CWARN ("LPX64 wrong length! strlen(%s)=%d != 18\n" , buf, len); +CERROR ("LPX64 wrong length! strlen(%s)=%d != 18\n" , buf, len); ret = -EINVAL ; } len =snprintf (buf,sizeof (buf),"%llu" , u64val);if (len !=20 ) { -CWARN ("LPU64 wrong length! strlen(%s)=%d != 20\n" , buf, len); +CERROR ("LPU64 wrong length! strlen(%s)=%d != 20\n" , buf, len); ret = -EINVAL ; } len =snprintf (buf,sizeof (buf),"%lld" , u64val);if (len !=2 ) { -CWARN ("LPD64 wrong length! strlen(%s)=%d != 2\n" , buf, len); +CERROR ("LPD64 wrong length! strlen(%s)=%d != 2\n" , buf, len); ret = -EINVAL ; }if ((u64val & ~PAGE_MASK ) >=PAGE_SIZE ) { -CWARN ("mask failed: u64val %llu >= %llu\n" , u64val, +CERROR ("mask failed: u64val %llu >= %llu\n" , u64val, (u64 )PAGE_SIZE ); ret = -EINVAL ; } +if (ret) +return ret;/* invalid string */ if (!test_string_to_size_err ("256B34" ,256 ,"B" , -EINVAL )) {
fs/lustre/obdclass/lprocfs_status.c +3 -3
@@ -236,7 +236,7 @@ static void obd_connect_data_seqprint(struct seq_file *m,* * - ``-EINVAL``: @buffer is not a proper numerical string * - ``-EOVERFLOW``: results does not fit into 64 bits. -* - ``-E2BIG ``: @buffer is not large (not a valid number) +* - ``-E2BIG ``: @buffer is too large (not a valid number) */ int string_to_size (u64 *size,const char *buffer,size_t count) {
@@ -1057,8 +1057,8 @@ int lprocfs_rd_connect_flags(struct seq_file *m, void *data)int rc; with_imp_locked(obd, imp, rc) { - flags = obd->u.cli.cl_import->imp_connect_data.ocd_connect_flags; - flags2 = obd->u.cli.cl_import->imp_connect_data.ocd_connect_flags2; + flags = imp->imp_connect_data.ocd_connect_flags; + flags2 = imp->imp_connect_data.ocd_connect_flags2;seq_printf (m,"flags=%#llx\n" , flags);seq_printf (m,"flags2=%#llx\n" , flags2);obd_connect_seq_flags2str (m, flags, flags2,"\n" );
fs/lustre/osc/osc_cache.c +1 -1
@@ -1655,7 +1655,7 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli,* run out of grants. In both cases we should write dirty pages out. * Adding a cache waiter will trigger urgent write-out no matter what * RPC size will be. -* The exiting condition (other then success) is no avail grants +* The exiting condition (other than success) is no avail grants * and no dirty pages caching, that really means there is no space * on the OST. */
fs/lustre/osc/osc_request.c +2 -3
@@ -1295,7 +1295,6 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, struct obdo *oa, u32 page_count, struct brw_page **pga, struct ptlrpc_request **reqp, - int reserve, int resend) { struct ptlrpc_request *req;
@@ -1948,7 +1947,7 @@ static int osc_brw_redo_request(struct ptlrpc_request *request, OST_WRITE) ? OBD_BRW_WRITE : OBD_BRW_READ, aa->aa_cli, aa->aa_oa, aa->aa_page_count, aa->aa_ppga, - &new_req, 0, 1); + &new_req, 1);if (rc)return rc;
@@ -2298,7 +2297,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, }sort_brw_pages (pga, page_count); - rc =osc_brw_prep_request (cmd, cli, oa, page_count, pga, &req,1 ,0 ); + rc =osc_brw_prep_request (cmd, cli, oa, page_count, pga, &req,0 );if (rc !=0 ) {CERROR ("prep_req failed: %d\n" , rc); goto out;
fs/lustre/ptlrpc/import.c +1 -1
@@ -1879,7 +1879,7 @@ timeout_t at_measured(struct adaptive_timeout *at, timeout_t timeout) at_min);if (at->at_current_timeout != old_timeout) CDEBUG(D_OTHER, - "AT %p change: old=%u new=%u delta=%d (val=%u) hist %u %u %u %u\n", + "AT %p change: old=%u new=%u delta=%d (val=%d) hist %u %u %u %u\n", at, old_timeout, at->at_current_timeout, at->at_current_timeout - old_timeout, timeout, at->at_hist[0], at->at_hist[1], at->at_hist[2],
Signature · 1 line
1.8.3.1
PATCH 4/45 lnet: merge lnet_md_alloc into lnet_md_build.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
lnet_md_alloc is only called twice, each time immediately before a call to lnet_md_build(), and these are the only calls to lnet_md_build().
So simplify the code by merging lnet_md_alloc into lnet_md_build.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13004
Lustre-commit: c0598f15dd502 ("LU-13004 lnet: merge lnet_md_alloc into lnet_md_build.")
Reviewed-on: https://review.whamcloud.com/37841
--- include/linux/lnet/lib-lnet.h | 39 ---------------------- net/lnet/lnet/lib-md.c | 75 +++++++++++++++++++++++++++++-------------- 2 files changed, 51 insertions(+), 63 deletions(-)
include/linux/lnet/lib-lnet.h +0 -39
@@ -215,45 +215,6 @@ static inline int lnet_md_unlinkable(struct lnet_libmd *md)#define MAX_PORTALS 64 -static inline struct lnet_libmd * -lnet_md_alloc (struct lnet_md *umd) -{ -struct lnet_libmd *md; -unsigned int size; -unsigned int niov; - -if (umd->options &LNET_MD_KIOV ) { - niov = umd->length; - size = offsetof(struct lnet_libmd, md_iov.kiov[niov]); - }else { - niov =1 ; - size = offsetof(struct lnet_libmd, md_iov.iov[niov]); - } - -if (size <=LNET_SMALL_MD_SIZE ) { - md =kmem_cache_alloc (lnet_small_mds_cachep, -GFP_NOFS | __GFP_ZERO); -if (md) { -CDEBUG (D_MALLOC , -"slab-alloced 'md' of size %u@%p.\n" , - size, md); - }else { -CDEBUG (D_MALLOC ,"failed to allocate 'md' of size %u\n" , - size); - } - }else { - md =kzalloc (size,GFP_NOFS ); - } -if (md) { -/* Set here in case of early free */ - md->md_options = umd->options; - md->md_niov = niov; -INIT_LIST_HEAD (&md->md_list); - } - -return md; -} -struct lnet_libhandle *lnet_res_lh_lookup (struct lnet_res_container *rec,u64 cookie); void lnet_res_lh_initialize(struct lnet_res_container *rec,
net/lnet/lnet/lib-md.c +51 -24
123 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 5/45 lnet: always put a page list into struct lnet_libmd
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
'struct lnet_libmd' is only created in lnet_md_build(). It can be given a list of pages or a virtual address. In the latter case, the memory will eventually be split into a list of pages. It is cleaner to split it into a list of pages early so that all lower levels only need to handle one type: a page list.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13004
Lustre-commit: 857f11169fc8 ("LU-13004 lnet: always put a page list into struct lnet_libmd")
Reviewed-on: https://review.whamcloud.com/37842
--- net/lnet/lnet/lib-md.c | 38 +++++++++++++++++++++++++------------- 1 file changed, 25 insertions(+), 13 deletions(-)
net/lnet/lnet/lib-md.c +25 -13
@@ -173,13 +173,12 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset)struct lnet_libmd *lmd;unsigned int size; -if ((umd->options &LNET_MD_KIOV ) !=0 ) { +if (umd->options &LNET_MD_KIOV ) niov = umd->length; - size = offsetof(struct lnet_libmd, md_iov.kiov[niov]); - }else { - niov =1 ; - size = offsetof(struct lnet_libmd, md_iov.iov[niov]); - } +else + niov =DIV_ROUND_UP (offset_in_page (umd->start) + umd->length, +PAGE_SIZE ); + size = offsetof(struct lnet_libmd, md_iov.kiov[niov]);if (size <=LNET_SMALL_MD_SIZE ) { lmd =kmem_cache_zalloc (lnet_small_mds_cachep,GFP_NOFS );
@@ -200,7 +199,6 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) lmd->md_niov = niov;INIT_LIST_HEAD (&lmd->md_list); - lmd->md_me =NULL ; lmd->md_start = umd->start; lmd->md_offset =0 ;
@@ -238,19 +236,33 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset)lnet_md_free (lmd);return ERR_PTR (-EINVAL ); } - }else {/* contiguous */ - lmd->md_length = umd->length; - niov =1 ; - lmd->md_niov =1 ; - lmd->md_iov.iov[0 ].iov_base = umd->start; - lmd->md_iov.iov[0 ].iov_len = umd->length; + }else {/* contiguous - split into pages */ +void *pa = umd->start; +int len = umd->length; + lmd->md_length = len; + i =0 ; +while (len) { +int plen; + + plen =min_t (int, len,PAGE_SIZE -offset_in_page (pa)); + + lmd->md_iov.kiov[i].bv_page = +lnet_kvaddr_to_page ((unsigned long ) pa); + lmd->md_iov.kiov[i].bv_offset =offset_in_page (pa); + lmd->md_iov.kiov[i].bv_len = plen; + + len -= plen; + pa += plen; + i +=1 ; + }if ((umd->options &LNET_MD_MAX_SIZE ) &&/* max size used */ (umd->max_size <0 || umd->max_size > (int )umd->length)) {/* illegal max_size */ lnet_md_free (lmd);return ERR_PTR (-EINVAL ); } + lmd->md_options |=LNET_MD_KIOV ; }return lmd;
Signature · 1 line
1.8.3.1
PATCH 6/45 lnet: discard kvec option from lnet_libmd.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
The 'struct kvec' field in 'struct lnet_libmd' is no longer set - the memory is always presented as pages. So remove that option and the union that contained iov and kiov.
Rename md_iov.kiov to md_kiov.
Discard all code that made use of md_iov.iov.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13004
Lustre-commit: 4775d52439f4f ("LU-13004 lnet: discard kvec option from lnet_libmd.")
Reviewed-on: https://review.whamcloud.com/37843
--- include/linux/lnet/lib-lnet.h | 7 +--- include/linux/lnet/lib-types.h | 5 +-- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 14 ++----- net/lnet/lnet/lib-md.c | 79 +++++++++---------------------------- net/lnet/lnet/lib-move.c | 5 +-- 5 files changed, 27 insertions(+), 83 deletions(-)
include/linux/lnet/lib-lnet.h +2 -5
@@ -89,7 +89,7 @@bool lnet_is_route_alive (struct lnet_route *route); -#define LNET_SMALL_MD_SIZE offsetof(struct lnet_libmd, md_iov.iov[1]) +#define LNET_SMALL_MD_SIZE offsetof(struct lnet_libmd, md_kiov[1])extern struct kmem_cache *lnet_mes_cachep;/* MEs kmem_cache */ extern struct kmem_cache *lnet_small_mds_cachep;/* <= LNET_SMALL_MD_SIZE bytes * MDs kmem_cache
@@ -165,10 +165,7 @@ static inline int lnet_md_unlinkable(struct lnet_libmd *md)LASSERTF (md->md_rspt_ptr ==NULL ,"md %p rsp %p\n" , md, md->md_rspt_ptr); -if ((md->md_options &LNET_MD_KIOV ) !=0 ) - size = offsetof(struct lnet_libmd, md_iov.kiov[md->md_niov]); -else - size = offsetof(struct lnet_libmd, md_iov.iov[md->md_niov]); + size = offsetof(struct lnet_libmd, md_kiov[md->md_niov]);if (size <=LNET_SMALL_MD_SIZE ) {CDEBUG (D_MALLOC ,"slab-freed 'md'@%p.\n" , md);
include/linux/lnet/lib-types.h +1 -4
@@ -208,10 +208,7 @@ struct lnet_libmd {struct lnet_rsp_tracker *md_rspt_ptr;lnet_handler_t md_handler;struct lnet_handle_md md_bulk_handle; -union { -struct kvec iov[LNET_MAX_IOV ]; -struct bio_vec kiov[LNET_MAX_IOV ]; - } md_iov; +struct bio_vec md_kiov[LNET_MAX_IOV ]; };#define LNET_MD_FLAG_ZOMBIE BIT(0)
net/lnet/klnds/o2iblnd/o2iblnd_cb.c +4 -10
@@ -1626,16 +1626,10 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, ibmsg = tx->tx_msg; rd = &ibmsg->ibm_u.get.ibgm_rd; -if (!(lntmsg->msg_md->md_options &LNET_MD_KIOV )) - rc =kiblnd_setup_rd_iov (ni, tx, rd, - lntmsg->msg_md->md_niov, - lntmsg->msg_md->md_iov.iov, -0 , lntmsg->msg_md->md_length); -else - rc =kiblnd_setup_rd_kiov (ni, tx, rd, - lntmsg->msg_md->md_niov, - lntmsg->msg_md->md_iov.kiov, -0 , lntmsg->msg_md->md_length); + rc =kiblnd_setup_rd_kiov (ni, tx, rd, + lntmsg->msg_md->md_niov, + lntmsg->msg_md->md_kiov, +0 , lntmsg->msg_md->md_length);if (rc) {CERROR ("Can't setup GET sink for %s: %d\n" ,libcfs_nid2str (target.nid), rc);
net/lnet/lnet/lib-md.c +19 -60
123 lines not on this page · raw message
net/lnet/lnet/lib-move.c +1 -4
@@ -538,10 +538,7 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,LASSERT (!msg->msg_kiov); msg->msg_niov = md->md_niov; -if (md->md_options &LNET_MD_KIOV ) - msg->msg_kiov = md->md_iov.kiov; -else - msg->msg_iov = md->md_iov.iov; + msg->msg_kiov = md->md_kiov; }void
Signature · 1 line
1.8.3.1
PATCH 7/45 lnet: remove msg_iov from lnet_msg.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
This field is never set, so remove it and remove all uses of it.
Conversely, msg_kiov is always non-NULL, so remove code that depends on it being NULL.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13004
Lustre-commit: 585e6df77ebf2 ("LU-13004 lnet: remove msg_iov from lnet_msg.")
Reviewed-on: https://review.whamcloud.com/37844
--- include/linux/lnet/lib-types.h | 1 - net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 34 ++++++++------------------------ net/lnet/klnds/socklnd/socklnd_cb.c | 39 +++++++++++-------------------------- net/lnet/lnet/lib-move.c | 16 ++++----------- net/lnet/lnet/lo.c | 17 +++++----------- 5 files changed, 28 insertions(+), 79 deletions(-)
include/linux/lnet/lib-types.h +0 -1
@@ -164,7 +164,6 @@ struct lnet_msg {unsigned int msg_wanted;unsigned int msg_offset;unsigned int msg_niov; -struct kvec *msg_iov;struct bio_vec *msg_kiov;struct lnet_event msg_ev;
net/lnet/klnds/o2iblnd/o2iblnd_cb.c +8 -26
@@ -1564,7 +1564,6 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,int target_is_router = lntmsg->msg_target_is_router;int routing = lntmsg->msg_routing;unsigned int payload_niov = lntmsg->msg_niov; -struct kvec *payload_iov = lntmsg->msg_iov;struct bio_vec *payload_kiov = lntmsg->msg_kiov;unsigned int payload_offset = lntmsg->msg_offset;unsigned int payload_nob = lntmsg->msg_len;
@@ -1585,17 +1584,10 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,/* Thread context */ LASSERT (!in_interrupt ()); -/* payload is either all vaddrs or all pages */ -LASSERT (!(payload_kiov && payload_iov)); -if (payload_kiov) -iov_iter_bvec (&from,WRITE , - payload_kiov, payload_niov, - payload_nob + payload_offset); -else -iov_iter_kvec (&from,WRITE , - payload_iov, payload_niov, - payload_nob + payload_offset); +iov_iter_bvec (&from,WRITE , + payload_kiov, payload_niov, + payload_nob + payload_offset);iov_iter_advance (&from, payload_offset);
@@ -1627,9 +1619,8 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, ibmsg = tx->tx_msg; rd = &ibmsg->ibm_u.get.ibgm_rd; rc =kiblnd_setup_rd_kiov (ni, tx, rd, - lntmsg->msg_md->md_niov, - lntmsg->msg_md->md_kiov, -0 , lntmsg->msg_md->md_length); + payload_niov, payload_kiov, + payload_offset, payload_nob);if (rc) {CERROR ("Can't setup GET sink for %s: %d\n" ,libcfs_nid2str (target.nid), rc);
@@ -1672,14 +1663,9 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,return -ENOMEM ; } -if (!payload_kiov) - rc =kiblnd_setup_rd_iov (ni, tx, tx->tx_rd, - payload_niov, payload_iov, - payload_offset, payload_nob); -else - rc =kiblnd_setup_rd_kiov (ni, tx, tx->tx_rd, - payload_niov, payload_kiov, - payload_offset, payload_nob); + rc =kiblnd_setup_rd_kiov (ni, tx, tx->tx_rd, + payload_niov, payload_kiov, + payload_offset, payload_nob);if (rc) {CERROR ("Can't setup PUT src for %s: %d\n" ,libcfs_nid2str (target.nid), rc);
@@ -1733,7 +1719,6 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, {struct lnet_process_id target = lntmsg->msg_target;unsigned int niov = lntmsg->msg_niov; -struct kvec *iov = lntmsg->msg_iov;struct bio_vec *kiov = lntmsg->msg_kiov;unsigned int offset = lntmsg->msg_offset;unsigned int nob = lntmsg->msg_len;
@@ -1749,9 +1734,6 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid,if (!nob) rc =0 ; -else if (!kiov) - rc =kiblnd_setup_rd_iov (ni, tx, tx->tx_rd, - niov, iov, offset, nob);else rc =kiblnd_setup_rd_kiov (ni, tx, tx->tx_rd, niov, kiov, offset, nob);
net/lnet/klnds/socklnd/socklnd_cb.c +11 -28
@@ -914,7 +914,6 @@ struct ksock_route *int type = lntmsg->msg_type;struct lnet_process_id target = lntmsg->msg_target;unsigned int payload_niov = lntmsg->msg_niov; -struct kvec *payload_iov = lntmsg->msg_iov;struct bio_vec *payload_kiov = lntmsg->msg_kiov;unsigned int payload_offset = lntmsg->msg_offset;unsigned int payload_nob = lntmsg->msg_len;
@@ -931,16 +930,10 @@ struct ksock_route *LASSERT (!payload_nob || payload_niov >0 );LASSERT (payload_niov <=LNET_MAX_IOV ); -/* payload is either all vaddrs or all pages */ -LASSERT (!(payload_kiov && payload_iov));LASSERT (!in_interrupt ()); -if (payload_iov) - desc_size = offsetof(struct ksock_tx , - tx_frags.virt.iov[1 + payload_niov]); -else - desc_size = offsetof(struct ksock_tx , - tx_frags.paged.kiov[payload_niov]); + desc_size = offsetof(struct ksock_tx , + tx_frags.paged.kiov[payload_niov]);if (lntmsg->msg_vmflush) mpflag =memalloc_noreclaim_save ();
@@ -956,25 +949,15 @@ struct ksock_route * tx->tx_conn =NULL ;/* set when assigned a conn */ tx->tx_lnetmsg = lntmsg; -if (payload_iov) { - tx->tx_kiov =NULL ; - tx->tx_nkiov =0 ; - tx->tx_iov = tx->tx_frags.virt.iov; - tx->tx_niov =1 + -lnet_extract_iov (payload_niov, &tx->tx_iov[1 ], - payload_niov, payload_iov, - payload_offset, payload_nob); - }else { - tx->tx_niov =1 ; - tx->tx_iov = &tx->tx_frags.paged.iov; - tx->tx_kiov = tx->tx_frags.paged.kiov; - tx->tx_nkiov =lnet_extract_kiov (payload_niov, tx->tx_kiov, - payload_niov, payload_kiov, - payload_offset, payload_nob); - -if (payload_nob >= *ksocknal_tunables.ksnd_zc_min_payload) - tx->tx_zc_capable =1 ; - } + tx->tx_niov =1 ; + tx->tx_iov = &tx->tx_frags.paged.iov; + tx->tx_kiov = tx->tx_frags.paged.kiov; + tx->tx_nkiov =lnet_extract_kiov (payload_niov, tx->tx_kiov, + payload_niov, payload_kiov, + payload_offset, payload_nob); + +if (payload_nob >= *ksocknal_tunables.ksnd_zc_min_payload) + tx->tx_zc_capable =1 ; tx->tx_msg.ksm_csum =0 ; tx->tx_msg.ksm_type =KSOCK_MSG_LNET ;
net/lnet/lnet/lib-move.c +4 -12
@@ -485,7 +485,6 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, unsigned int rlen) {unsigned int niov =0 ; -struct kvec *iov =NULL ;struct bio_vec *kiov =NULL ;struct iov_iter to;int rc;
@@ -505,21 +504,16 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,if (mlen) { niov = msg->msg_niov; - iov = msg->msg_iov; kiov = msg->msg_kiov;LASSERT (niov >0 ); -LASSERT (!iov != !kiov); +LASSERT (kiov); } } -if (iov) { -iov_iter_kvec (&to,READ , iov, niov, mlen + offset); -iov_iter_advance (&to, offset); - }else { -iov_iter_bvec (&to,READ , kiov, niov, mlen + offset); -iov_iter_advance (&to, offset); - } +iov_iter_bvec (&to,READ , kiov, niov, mlen + offset); +iov_iter_advance (&to, offset); + rc = ni->ni_net->net_lnd->lnd_recv (ni, private, msg, delayed, &to, rlen);if (rc <0 )lnet_finalize (msg, rc);
@@ -534,7 +528,6 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,LASSERT (!msg->msg_routing);LASSERT (md);LASSERT (!msg->msg_niov); -LASSERT (!msg->msg_iov);LASSERT (!msg->msg_kiov); msg->msg_niov = md->md_niov;
@@ -809,7 +802,6 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,struct lnet_rtrbufpool *rbp;struct lnet_rtrbuf *rb; -LASSERT (!msg->msg_iov);LASSERT (!msg->msg_kiov);LASSERT (!msg->msg_niov);LASSERT (msg->msg_routing);
net/lnet/lnet/lo.c +5 -12
@@ -51,18 +51,11 @@struct lnet_msg *sendmsg = private;if (lntmsg) {/* not discarding */ -if (sendmsg->msg_iov) -lnet_copy_iov2iter (to, - sendmsg->msg_niov, - sendmsg->msg_iov, - sendmsg->msg_offset, -iov_iter_count (to)); -else -lnet_copy_kiov2iter (to, - sendmsg->msg_niov, - sendmsg->msg_kiov, - sendmsg->msg_offset, -iov_iter_count (to)); +lnet_copy_kiov2iter (to, + sendmsg->msg_niov, + sendmsg->msg_kiov, + sendmsg->msg_offset, +iov_iter_count (to));lnet_finalize (lntmsg,0 ); }
Signature · 1 line
1.8.3.1
PATCH 8/45 lnet: o2iblnd: discard kiblnd_setup_rd_iov
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
The 'to' passed to lnd_recv is *always* ITER_BVEC, so we can discard kiblnd_setup_rd_iov().
WC-bug-id: https://jira.whamcloud.com/browse/LU-13004
Lustre-commit: e35b7751f49fb ("LU-13004 lnet: remove the 'struct kvec' arg from lnd_send")
Reviewed-on: https://review.whamcloud.com/37845
--- include/linux/lnet/lib-types.h | 6 +-- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 80 ++----------------------------------- 2 files changed, 5 insertions(+), 81 deletions(-)
include/linux/lnet/lib-types.h +1 -5
@@ -239,11 +239,7 @@ struct lnet_lnd {/* * In data movement APIs below, payload buffers are described as a set -* of 'niov' fragments which are... -* EITHER -* in virtual memory (struct iovec *iov != NULL) -* OR -* in pages (kernel only: plt_kiov_t *kiov != NULL). +* of 'niov' fragments which are in pages. * The LND may NOT overwrite these fragment descriptors. * An 'offset' and may specify a byte offset within the set of * fragments to start from
net/lnet/klnds/o2iblnd/o2iblnd_cb.c +4 -76
98 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 9/45 lustre: ptlrpc: return proper write count from ping_store
In reply to James SimmonsFrom: Oleg Drokin <[email protected]>
If we return 0 as a measure of success, that is actually taken by userspace as "we wrote 0 bytes, but there's no error, so they want us to retry". So tell them we ate all their data instead to make them happy.
Interesting that when you do shell-redirect it enters the retry loop, but badarea_io does not.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13470
Lustre-commit: 16d62976d212d ("LU-13470 ptlrpc: return proper write count from ping_store")
Reviewed-on: https://review.whamcloud.com/38304
--- fs/lustre/ptlrpc/lproc_ptlrpc.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
fs/lustre/ptlrpc/lproc_ptlrpc.c +3 -1
@@ -1271,7 +1271,9 @@ ssize_t ping_show(struct kobject *kobj, struct attribute *attr,ssize_t ping_store (struct kobject *kobj,struct attribute *attr,const char *buffer,size_t count) { -return ping_show (kobj, attr, (char *)buffer); +int rc =ping_show (kobj, attr, (char *)buffer); + +return (rc <0 ) ? rc : count; }EXPORT_SYMBOL (ping_store);
Signature · 1 line
1.8.3.1
PATCH 10/45 lustre: sec: check permissions for changelogs access
In reply to James SimmonsFrom: Sebastien Buisson <[email protected]>
root permissions should be checked when reading or clearing changelogs from clients. In particular, if root is squashed via a nodemap entry, it should not be allowed to access changelogs.
To achieve this send mdt body along with RQF_LLOG_ORIGIN_HANDLE_CREATE and RQF_MDT_SET_INFO requests. And on server side, retrieve user credentials and make sure they have root permission.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13064
Lustre-commit: 4e8fcee92d751 ("LU-13064 sec: check permissions for changelogs access")
Reviewed-on: https://review.whamcloud.com/36990
--- fs/lustre/include/lustre_req_layout.h | 1 + fs/lustre/ptlrpc/layout.c | 15 ++++++++++++++- fs/lustre/ptlrpc/llog_client.c | 4 ++++ fs/lustre/ptlrpc/pack_generic.c | 7 ++++++- fs/lustre/ptlrpc/ptlrpc_internal.h | 19 +++++++++++++++++++ 5 files changed, 44 insertions(+), 2 deletions(-)
fs/lustre/include/lustre_req_layout.h +1 -0
@@ -123,6 +123,7 @@ void req_capsule_shrink(struct req_capsule *pill,extern struct req_format RQF_OBD_PING ;extern struct req_format RQF_OBD_SET_INFO ; +extern struct req_format RQF_MDT_SET_INFO ;extern struct req_format RQF_SEC_CTX ;/* MGS req_format */ extern struct req_format RQF_MGS_TARGET_REG ;
fs/lustre/ptlrpc/layout.c +14 -1
@@ -348,6 +348,13 @@ &RMF_SETINFO_VAL }; +static const struct req_msg_field *mdt_set_info_client[] = { + &RMF_PTLRPC_BODY , + &RMF_SETINFO_KEY , + &RMF_SETINFO_VAL , + &RMF_MDT_BODY +}; +static const struct req_msg_field *ost_grant_shrink_client[] = { &RMF_PTLRPC_BODY , &RMF_SETINFO_KEY ,
@@ -549,7 +556,8 @@static const struct req_msg_field *llog_origin_handle_create_client[] = { &RMF_PTLRPC_BODY , &RMF_LLOGD_BODY , - &RMF_NAME + &RMF_NAME , + &RMF_MDT_BODY , };static const struct req_msg_field *llogd_body_only[] = {
@@ -698,6 +706,7 @@static struct req_format *req_formats[] = { &RQF_OBD_PING , &RQF_OBD_SET_INFO , + &RQF_MDT_SET_INFO , &RQF_SEC_CTX , &RQF_MGS_TARGET_REG , &RQF_MGS_CONFIG_READ ,
@@ -1238,6 +1247,10 @@ struct req_format RQF_OBD_SET_INFO =DEFINE_REQ_FMT0 ("OBD_SET_INFO" , obd_set_info_client, empty);EXPORT_SYMBOL (RQF_OBD_SET_INFO ); +struct req_format RQF_MDT_SET_INFO = +DEFINE_REQ_FMT0 ("MDT_SET_INFO" , mdt_set_info_client, empty); +EXPORT_SYMBOL (RQF_MDT_SET_INFO ); +struct req_format RQF_SEC_CTX =DEFINE_REQ_FMT0 ("SEC_CTX" , empty, empty);EXPORT_SYMBOL (RQF_SEC_CTX );
fs/lustre/ptlrpc/llog_client.c +4 -0
@@ -44,6 +44,8 @@#include <lustre_net.h> #include <linux/list.h> +#include "ptlrpc_internal.h" +#define LLOG_CLIENT_ENTRY (ctxt, imp) do { \ mutex_lock(&ctxt->loc_mutex); \ if (ctxt->loc_imp) { \
@@ -120,6 +122,8 @@ static int llog_client_open(const struct lu_env *env, strlen(name) + 1);LASSERT (tmp);strcpy (tmp, name); + +do_pack_body (req); } rc =ptlrpc_queue_wait (req);
fs/lustre/ptlrpc/pack_generic.c +6 -1
@@ -1506,7 +1506,9 @@ int do_set_info_async(struct obd_import *imp,char *tmp;int rc; - req =ptlrpc_request_alloc (imp, &RQF_OBD_SET_INFO ); + req =ptlrpc_request_alloc (imp,KEY_IS (KEY_CHANGELOG_CLEAR ) ? + &RQF_MDT_SET_INFO : + &RQF_OBD_SET_INFO );if (!req)return -ENOMEM ;
@@ -1520,6 +1522,9 @@ int do_set_info_async(struct obd_import *imp,return rc; } +if (KEY_IS (KEY_CHANGELOG_CLEAR )) +do_pack_body (req); + tmp =req_capsule_client_get (&req->rq_pill, &RMF_SETINFO_KEY );memcpy (tmp, key, keylen); tmp =req_capsule_client_get (&req->rq_pill, &RMF_SETINFO_VAL );
fs/lustre/ptlrpc/ptlrpc_internal.h +19 -0
@@ -377,4 +377,23 @@ static inline bool ptlrpc_req_is_disconnect(struct ptlrpc_request *req)return false; } +static inline void do_pack_body (struct ptlrpc_request *req) +{ +struct mdt_body *b =req_capsule_client_get (&req->rq_pill, + &RMF_MDT_BODY ); + +if (!b) +return ; + + b->mbo_valid =0 ; + b->mbo_eadatasize =0 ; + b->mbo_flags =0 ; + b->mbo_suppgid =-1 ; + b->mbo_uid =from_kuid (&init_user_ns,current_uid ()); + b->mbo_gid =from_kgid (&init_user_ns,current_gid ()); + b->mbo_fsuid =from_kuid (&init_user_ns,current_fsuid ()); + b->mbo_fsgid =from_kgid (&init_user_ns,current_fsgid ()); + b->mbo_capability =current_cap ().cap[0 ]; +} +#endif /* PTLRPC_INTERNAL_H */
Signature · 1 line
1.8.3.1
PATCH 11/45 lustre: uapi: add OBD_CONNECT2_FIDMAP
In reply to James SimmonsFrom: Lai Siyao <[email protected]>
Add OBD_CONNECT2_FIDMAP connect flag, which indicates whether client supports FID mapping.
WC-bug-id: https://jira.whamcloud.com/browse/LU-11025
Lustre-commit: c96fa612d5b0f ("LU-11025 uapi: add OBD_CONNECT2_FIDMAP")
Reviewed-on: https://review.whamcloud.com/38232
--- fs/lustre/ptlrpc/wiretest.c | 2 ++ include/uapi/linux/lustre/lustre_idl.h | 1 + 2 files changed, 3 insertions(+)
fs/lustre/ptlrpc/wiretest.c +2 -0
@@ -1241,6 +1241,8 @@ void lustre_assert_wire_constants(void) OBD_CONNECT2_ASYNC_DISCARD);LASSERTF (OBD_CONNECT2_ENCRYPT ==0x8000ULL ,"found 0x%.16llxULL\n" ,OBD_CONNECT2_ENCRYPT ); +LASSERTF (OBD_CONNECT2_FIDMAP ==0x10000ULL ,"found 0x%.16llxULL\n" , +OBD_CONNECT2_FIDMAP );LASSERTF (OBD_CKSUM_CRC32 ==0x00000001UL ,"found 0x%.8xUL\n" , (unsigned int )OBD_CKSUM_CRC32 ); LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n",
include/uapi/linux/lustre/lustre_idl.h +1 -0
@@ -818,6 +818,7 @@ struct ptlrpc_body_v2 {* discard */ #define OBD_CONNECT2_ENCRYPT 0x8000ULL/* client-to-disk encrypt */ +#define OBD_CONNECT2_FIDMAP 0x10000ULL/* FID map */ /* XXX README XXX: * Please DO NOT add flag values here before first ensuring that this same * flag value is not in use on some other branch. Please clear any such
Signature · 1 line
1.8.3.1
PATCH 12/45 lustre: lov: lov_io_sub_init()) ASSERTION
In reply to James SimmonsFrom: Bobi Jam <[email protected]>
The assertion is_index_within_mirror() in lov_io_sub_init() should only be applied for a FLR file, since a plain file does not initialize the relevant FLR fields of the layout structure.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13429
Lustre-commit: 53274fbd4bcf4 ("LU-13429 lov: lov_io_sub_init()) ASSERTION")
Reviewed-on: https://review.whamcloud.com/38169
--- fs/lustre/lov/lov_io.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
fs/lustre/lov/lov_io.c +3 -1
@@ -113,7 +113,9 @@ static int lov_io_sub_init(const struct lu_env *env, struct lov_io *lio, !lov_r0(lov, index)->lo_sub[stripe]))return -EIO ; -LASSERTF (is_index_within_mirror (lov, index, lio->lis_mirror_index), +LASSERTF (ergo (lov_is_flr (lov), +is_index_within_mirror (lov, index, + lio->lis_mirror_index)),DFID "iot = %d, index = %d, mirror = %d\n" ,PFID (lu_object_fid (lov2lu (lov))), io->ci_type, index, lio->lis_mirror_index);
Signature · 1 line
1.8.3.1
PATCH 13/45 lnet: Introduce constant for the lolnd NID
In reply to James SimmonsFrom: Chris Horn <[email protected]>
This patch adds a new constant, LNET_NID_LO_0, to represent the lolnd NID 0 at lo.
HPE-bug-id: LUS-8457
WC-bug-id: https://jira.whamcloud.com/browse/LU-12222
Lustre-commit: 56203e4ba0a6 ("LU-12222 lnet: Introduce constant for the lolnd NID")
Reviewed-on: https://review.whamcloud.com/38312
--- fs/lustre/llite/llite_lib.c | 2 +- fs/lustre/lmv/lmv_obd.c | 4 ++-- fs/lustre/ptlrpc/events.c | 2 +- include/uapi/linux/lnet/lnet-types.h | 3 +++ net/lnet/lnet/api-ni.c | 6 +++--- net/lnet/lnet/config.c | 3 +-- net/lnet/lnet/lib-move.c | 6 +++--- net/lnet/lnet/lib-msg.c | 5 ++--- net/lnet/lnet/peer.c | 6 +++--- net/lnet/lnet/router.c | 2 +- 10 files changed, 20 insertions(+), 19 deletions(-)
fs/lustre/llite/llite_lib.c +1 -1
@@ -2817,7 +2817,7 @@ void ll_compute_rootsquash_state(struct ll_sb_info *sbi) i =0 ;while (LNetGetId (i++, &id) != -ENOENT ) { -if (LNET_NETTYP (LNET_NIDNET (id.nid)) ==LOLND ) +if (id.nid ==LNET_NID_LO_0 )continue ;if (cfs_match_nid (id.nid, &squash->rsi_nosquash_nids)) { matched = true;
fs/lustre/lmv/lmv_obd.c +2 -2
@@ -1114,7 +1114,7 @@ static int lmv_setup(struct obd_device *obd, struct lustre_cfg *lcfg)* can distribute subdirs evenly from the beginning. */ while (LNetGetId (i++, &lnet_id) != -ENOENT ) { -if (LNET_NETTYP (LNET_NIDNET (lnet_id.nid)) !=LOLND ) { +if (lnet_id.nid !=LNET_NID_LO_0 ) { lmv->lmv_qos_rr_index = (u32 )lnet_id.nid;break ; }
@@ -1208,7 +1208,7 @@ static int lmv_select_statfs_mdt(struct lmv_obd *lmv, u32 flags)if (LNetGetId (i, &lnet_id) == -ENOENT )break ; -if (LNET_NETTYP (LNET_NIDNET (lnet_id.nid)) !=LOLND ) { +if (lnet_id.nid !=LNET_NID_LO_0 ) {/* We dont need a full 64-bit modulus, just enough * to distribute the requests across MDTs evenly. */
fs/lustre/ptlrpc/events.c +1 -1
@@ -478,7 +478,7 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid,continue ;if (dist ==0 ) {/* local! use loopback LND */ - peer->nid = *self =LNET_MKNID (LNET_MKNET (LOLND ,0 ),0 ); + peer->nid = *self =LNET_NID_LO_0 ; rc =0 ;break ; }
include/uapi/linux/lnet/lnet-types.h +3 -0
@@ -109,6 +109,9 @@ static inline __u32 LNET_MKNET(__u32 type, __u32 num)return (type <<16 ) | num; } +/** The lolnd NID (i.e. myself) */ +#define LNET_NID_LO_0 LNET_MKNID(LNET_MKNET(LOLND, 0), 0) +/* Packed version of lnet_process_id to transfer via network */ struct lnet_process_id_packed {/* node id / process id */
net/lnet/lnet/api-ni.c +3 -3
@@ -1567,7 +1567,7 @@ struct lnet_ping_buffer */* Loopback is guaranteed to be present */ if (pinfo->pi_nnis <1 || pinfo->pi_nnis > lnet_interfaces_max)return -ERANGE ; -if (LNET_NETTYP (LNET_NIDNET (LNET_PING_INFO_LONI (pinfo))) !=LOLND ) +if (LNET_PING_INFO_LONI (pinfo) !=LNET_NID_LO_0 )return -EPROTO ;return 0 ; }
@@ -2732,7 +2732,7 @@ void lnet_lib_exit(void) } cfg_ni->lic_nid = ni->ni_nid; -if (LNET_NETTYP (LNET_NIDNET (ni->ni_nid)) ==LOLND ) +if (ni->ni_nid ==LNET_NID_LO_0 ) cfg_ni->lic_status =LNET_NI_STATUS_UP ;else cfg_ni->lic_status = ni->ni_status->ns_status;
@@ -2824,7 +2824,7 @@ void lnet_lib_exit(void) config->cfg_config_u.cfg_net.net_peer_rtr_credits = ni->ni_net->net_tunables.lct_peer_rtr_credits; -if (LNET_NETTYP (LNET_NIDNET (ni->ni_nid)) ==LOLND ) +if (ni->ni_nid ==LNET_NID_LO_0 ) net_config->ni_status =LNET_NI_STATUS_UP ;else net_config->ni_status = ni->ni_status->ns_status;
net/lnet/lnet/config.c +1 -2
@@ -1179,8 +1179,7 @@ struct lnet_ni * goto token_error; nid =libcfs_str2nid (ltb->ltb_text); -if (nid ==LNET_NID_ANY || -LNET_NETTYP (LNET_NIDNET (nid)) ==LOLND ) +if (nid ==LNET_NID_ANY || nid ==LNET_NID_LO_0 ) goto token_error; } }
net/lnet/lnet/lib-move.c +3 -3
@@ -563,7 +563,7 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,int rc;LASSERT (!in_interrupt ()); -LASSERT (LNET_NETTYP (LNET_NIDNET (ni->ni_nid)) ==LOLND || +LASSERT (ni->ni_nid ==LNET_NID_LO_0 || (msg->msg_txcredit && msg->msg_peertxcredit)); rc = ni->ni_net->net_lnd->lnd_send (ni, priv, msg);
@@ -2407,7 +2407,7 @@ struct lnet_ni **/ send_data.sd_msg = msg; send_data.sd_cpt = cpt; -if (LNET_NETTYP (LNET_NIDNET (dst_nid)) ==LOLND ) { +if (dst_nid ==LNET_NID_LO_0 ) { rc =lnet_handle_lo_send (&send_data);lnet_net_unlock (cpt);return rc;
@@ -4793,7 +4793,7 @@ struct lnet_msg *if (srcnidp) *srcnidp = dstnid;if (orderp) { -if (LNET_NETTYP (LNET_NIDNET (dstnid)) ==LOLND ) +if (dstnid ==LNET_NID_LO_0 ) *orderp =0 ;else *orderp =1 ;
net/lnet/lnet/lib-msg.c +2 -3
@@ -784,11 +784,10 @@/* if we're sending to the LOLND then the msg_txpeer will not be * set. So no need to sanity check it. */ -if (msg->msg_tx_committed && -LNET_NETTYP (LNET_NIDNET (msg->msg_txni->ni_nid)) !=LOLND ) +if (msg->msg_tx_committed && msg->msg_txni->ni_nid !=LNET_NID_LO_0 )LASSERT (msg->msg_txpeer);else if (msg->msg_tx_committed && -LNET_NETTYP (LNET_NIDNET (msg->msg_txni->ni_nid)) ==LOLND ) + msg->msg_txni->ni_nid ==LNET_NID_LO_0 ) lo = true; if (hstatus != LNET_MSG_STATUS_OK &&
net/lnet/lnet/peer.c +3 -3
@@ -233,7 +233,7 @@* to ever use a different interface when sending messages to * myself. */ -if (LNET_NETTYP (LNET_NIDNET (nid)) ==LOLND ) +if (nid ==LNET_NID_LO_0 ) lp->lp_state =LNET_PEER_NO_DISCOVERY ; lp->lp_cpt =lnet_nid_cpt_hash (nid,LNET_CPT_NUMBER );
@@ -2615,7 +2615,7 @@ static int lnet_peer_merge_data(struct lnet_peer *lp,* present in curnis[] then this peer is for this node. */ for (i =0 ; i < ncurnis; i++) { -if (LNET_NETTYP (LNET_NIDNET (curnis[i])) ==LOLND ) +if (curnis[i] ==LNET_NID_LO_0 )continue ;for (j =1 ; j < pbuf->pb_info.pi_nnis; j++) {if (curnis[i] == pbuf->pb_info.pi_ni[j].ns_nid) {
@@ -2838,7 +2838,7 @@ static int lnet_peer_data_present(struct lnet_peer *lp)if (pbuf->pb_info.pi_nnis <=1 ) goto out; nid = pbuf->pb_info.pi_ni[1 ].ns_nid; -if (LNET_NETTYP (LNET_NIDNET (lp->lp_primary_nid)) ==LOLND ) { +if (lp->lp_primary_nid ==LNET_NID_LO_0 ) { rc =lnet_peer_set_primary_nid (lp, nid, flags);if (!rc) rc =lnet_peer_merge_data (lp, pbuf);
net/lnet/lnet/router.c +1 -1
@@ -661,7 +661,7 @@ static void lnet_shuffle_seed(void) libcfs_net2str(net), hops, priority, libcfs_nid2str(gateway)); if (gateway == LNET_NID_ANY || - LNET_NETTYP(LNET_NIDNET(gateway)) == LOLND || + gateway == LNET_NID_LO_0 || net == LNET_NIDNET(LNET_NID_ANY) || LNET_NETTYP(net) == LOLND || LNET_NIDNET(gateway) == net ||
Signature · 1 line
1.8.3.1
PATCH 14/45 lustre: Remove inappropriate uses of BIT() macro.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
The BIT() macro exists for identifying a specific bit in a word when it is being used as a bitmap (or mask for set of flags etc). While it uses "1 << ...." it is not a general replacement for that construct and should not be used to simply to raise '2' to some power.
Varous places in lustre and libcfs use BIT() when a size, rather than a BIT, are required. Convert these to explicitly use "1 << exponent".
WC-bug-id: https://jira.whamcloud.com/browse/LU-6142
Lustre-commit: 423ad81446973 ("LU-6142 lustre: Remove inappropriate uses of BIT() macro.")
Reviewed-on: https://review.whamcloud.com/38373
--- fs/lustre/ldlm/ldlm_resource.c | 4 ++-- fs/lustre/llite/lproc_llite.c | 4 ++-- fs/lustre/obdclass/class_obd.c | 12 +++++++----- net/lnet/libcfs/module.c | 17 ++++++++--------- 4 files changed, 19 insertions(+), 18 deletions(-)
fs/lustre/ldlm/ldlm_resource.c +2 -2
@@ -613,8 +613,8 @@ struct ldlm_namespace *ldlm_namespace_new(struct obd_device *obd, char *name, ns->ns_bucket_bits = ldlm_ns_hash_defs[ns_type].nsd_all_bits - ldlm_ns_hash_defs[ns_type].nsd_bkt_bits; - ns->ns_rs_buckets =kvmalloc (BIT (ns->ns_bucket_bits) * -sizeof (ns->ns_rs_buckets[0 ]), + ns->ns_rs_buckets =kvzalloc ((1 << ns->ns_bucket_bits) * +sizeof (*ns->ns_rs_buckets),GFP_KERNEL );if (!ns->ns_rs_buckets) goto out_hash;
fs/lustre/llite/lproc_llite.c +2 -2
@@ -1780,7 +1780,7 @@ static void ll_display_extents_info(struct ll_rw_extents_info *io_extents, w = pp_info->pp_w_hist.oh_buckets[i]; read_cum += r; write_cum += w; - end =BIT (i +LL_HIST_START - units); + end =1 << (i +LL_HIST_START - units); seq_printf(seq, "%4lu%c - %4lu%c%c: %14lu %4u %4u | %14lu %4u %4u\n", start, *unitp, end, *unitp,
@@ -1958,7 +1958,7 @@ void ll_rw_stats_tally(struct ll_sb_info *sbi, pid_t pid,lprocfs_oh_clear (&io_extents->pp_extents[cur].pp_w_hist); } -for (i =0 ; (count >=BIT (LL_HIST_START + i)) && +for (i =0 ; (count >=1 << (LL_HIST_START + i)) && (i < (LL_HIST_MAX -1 )); i++) ;if (rw ==0 ) {
fs/lustre/obdclass/class_obd.c +7 -5
@@ -113,27 +113,29 @@ static int class_resolve_dev_name(u32 len, const char *name)static int obd_ioctl_is_invalid (struct obd_ioctl_data *data) { -if (data->ioc_len >BIT (30 )) { +const int maxlen =1 <<30 ; + +if (data->ioc_len > maxlen) {CERROR ("OBD ioctl: ioc_len larger than 1<<30\n" );return 1 ; } -if (data->ioc_inllen1 >BIT (30 )) { +if (data->ioc_inllen1 > maxlen) {CERROR ("OBD ioctl: ioc_inllen1 larger than 1<<30\n" );return 1 ; } -if (data->ioc_inllen2 >BIT (30 )) { +if (data->ioc_inllen2 > maxlen) {CERROR ("OBD ioctl: ioc_inllen2 larger than 1<<30\n" );return 1 ; } -if (data->ioc_inllen3 >BIT (30 )) { +if (data->ioc_inllen3 > maxlen) {CERROR ("OBD ioctl: ioc_inllen3 larger than 1<<30\n" );return 1 ; } -if (data->ioc_inllen4 >BIT (30 )) { +if (data->ioc_inllen4 > maxlen) {CERROR ("OBD ioctl: ioc_inllen4 larger than 1<<30\n" );return 1 ; }
net/lnet/libcfs/module.c +8 -9
@@ -80,18 +80,17 @@ static inline size_t libcfs_ioctl_packlen(struct libcfs_ioctl_data *data)static inline bool libcfs_ioctl_is_invalid (struct libcfs_ioctl_data *data) { -if (data->ioc_hdr.ioc_len >BIT (30 )) { -CERROR ("LIBCFS ioctl: ioc_len larger than 1<<30\n" ); +const int maxlen =1 <<30 ; + +if (data->ioc_hdr.ioc_len > maxlen)return true; - } -if (data->ioc_inllen1 >BIT (30 )) { -CERROR ("LIBCFS ioctl: ioc_inllen1 larger than 1<<30\n" ); + +if (data->ioc_inllen1 > maxlen)return true; - } -if (data->ioc_inllen2 >BIT (30 )) { -CERROR ("LIBCFS ioctl: ioc_inllen2 larger than 1<<30\n" ); + +if (data->ioc_inllen2 > maxlen)return true; - } +if (data->ioc_inlbuf1 && !data->ioc_inllen1) {CERROR ("LIBCFS ioctl: inlbuf1 pointer but 0 length\n" );return true;
Signature · 1 line
1.8.3.1
PATCH 15/45 lustre: mgc: protect from NULL exp in mgc_enqueue()
In reply to James SimmonsFrom: Sebastien Buisson <[email protected]>
Return an error in mgc_enqueue() is exp parameter is NULL. Otherwise, it might crash in class_exp2cliimp().
WC-bug-id: https://jira.whamcloud.com/browse/LU-13466
Lustre-commit: 83906f97c1d3b ("LU-13466 mgc: protect from NULL exp in mgc_enqueue()")
Reviewed-on: https://review.whamcloud.com/38280
--- fs/lustre/mgc/mgc_request.c | 3 +++ 1 file changed, 3 insertions(+)
fs/lustre/mgc/mgc_request.c +3 -0
@@ -875,6 +875,9 @@ static int mgc_enqueue(struct obd_export *exp, u32 type,int short_limit =cld_is_sptlrpc (cld);int rc; +if (!exp) +return -EBADR ; +CDEBUG (D_MGC ,"Enqueue for %s (res %#llx)\n" , cld->cld_logname, cld->cld_resid.name[0 ]);
Signature · 1 line
1.8.3.1
PATCH 16/45 lustre: llite: do not flush COW pages from mapping
In reply to James SimmonsFrom: Bobi Jam <[email protected]>
Do not flush COW pages from an executable mapping, as this will cause running binaries to segfault when mapped pages disappear under them.
This was modified to allow non-blocking PCC caching, but triggers immediate segfaults for binaries when the DLM locks are revoked, even when PCC is not in use.
The added test case failed 100% without this patch.
This patch is only a partial fix to avoid the segfaults. There still needs to be something done to re-introduce the PCC functionality, but should be conditional on PCC enabled, and not for running binaries.
There is a problem for PCC when multiple clients read/write on a shared mmapped file. We exclude mmap_sanity tst6 in test_4 from PCC testing temporarily.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13137
Lustre-commit: 13a0066afb8d8 ("LU-13137 llite: do not flush COW pages from mapping")
Reviewed-on: https://review.whamcloud.com/37278
--- fs/lustre/llite/llite_mmap.c | 2 +- fs/lustre/llite/vvp_object.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
fs/lustre/llite/llite_mmap.c +1 -1
@@ -519,7 +519,7 @@ int ll_teardown_mmaps(struct address_space *mapping, u64 first, u64 last)if (mapping_mapped (mapping)) { rc =0 ;unmap_mapping_range (mapping, first +PAGE_SIZE -1 , - last - first +1 ,1 ); + last - first +1 ,0 ); }return rc;
fs/lustre/llite/vvp_object.c +1 -1
@@ -146,7 +146,7 @@ static int vvp_conf_set(const struct lu_env *env, struct cl_object *obj,* a price themselves. */ unmap_mapping_range (conf->coc_inode->i_mapping, -0 ,OBD_OBJECT_EOF ,1 ); +0 ,OBD_OBJECT_EOF ,0 );pcc_layout_invalidate (conf->coc_inode); }
Signature · 1 line
1.8.3.1
PATCH 17/45 lustre: quota: quota pools for OSTs
In reply to James SimmonsFrom: Sergey Cheremencev <[email protected]>
Patch allows to apply quota settings
not only for the whole system, but also
for different OST pools. Since this patch each "LOD" pool is duplicated by QMT.
Thus quota pools(QP) could be tuned by standard lctl pool_new/add/remove/erase commands. All QPs are subset of a global pool that includes all data devices in a system, including DOM. However DOM is not supported. I don't see a lot of work to add DOM support in future - just need to decide how MDTs could be combined in a pool.
The main idea of QP is to find all pools
for requested ID(usr/grp/prj) and apply minimum limit. The patch doesn't affect
qsd side, so slaves know nothing about
pools and different limits. Qunit and edquot are calculated for each slave on master.
To apply quota on QP, the patch adds key "-o" to lfs setquota. To get quotas for QP, it provides long option "--pool" in lfs quota. See examples of using in sanity-quota_1b/c/d.
Now QPs work properly only on a clean system. Support of recalculation granted space in case of adding/removing OSTs in a pool will be added in the next patch together with accounting already granted space by each ID in a POOl.
WC-bug-id: https://jira.whamcloud.com/browse/LU-11023
Lustre-commit: 09f9fb3211cd9 ("LU-11023 quota: quota pools for OSTs")
Reviewed-on: https://review.whamcloud.com/35615
--- fs/lustre/include/lu_object.h | 11 +------ fs/lustre/include/lustre_req_layout.h | 1 + fs/lustre/include/lustre_swab.h | 2 +- fs/lustre/include/obd_target.h | 54 +++++++++++++++++++++++++++++++++ fs/lustre/llite/dir.c | 30 +++++++++++++++--- fs/lustre/mdc/mdc_request.c | 40 ++++++++++++++++-------- fs/lustre/ptlrpc/layout.c | 6 ++-- fs/lustre/ptlrpc/pack_generic.c | 7 ++++- include/uapi/linux/lustre/lustre_idl.h | 21 ++++++++----- include/uapi/linux/lustre/lustre_user.h | 11 +++++++ 10 files changed, 143 insertions(+), 40 deletions(-) create mode 100644 fs/lustre/include/obd_target.h
fs/lustre/include/lu_object.h +1 -10
@@ -38,6 +38,7 @@#include <linux/percpu_counter.h> #include <linux/libcfs/libcfs.h> #include <linux/ctype.h> +#include <obd_target.h> #include <uapi/linux/lustre/lustre_idl.h> #include <lu_ref.h>
@@ -1399,16 +1400,6 @@ struct lu_kmem_descr {extern u32 lu_context_tags_default;extern u32 lu_session_tags_default; -/* Generic subset of tgts */ -struct lu_tgt_pool { -u32 *op_array;/* array of index of -* lov_obd->lov_tgts -*/ -unsigned int op_count;/* number of tgts in the array */ -unsigned int op_size;/* allocated size of op_array */ -struct rw_semaphore op_rw_sem;/* to protect lu_tgt_pool use */ -}; -/* round-robin QoS data for LOD/LMV */ struct lu_qos_rr {spinlock_t lqr_alloc;/* protect allocation index */
fs/lustre/include/lustre_req_layout.h +1 -0
@@ -266,6 +266,7 @@ void req_capsule_shrink(struct req_capsule *pill,extern struct req_msg_field RMF_CAPA2 ;extern struct req_msg_field RMF_OBD_QUOTACHECK ;extern struct req_msg_field RMF_OBD_QUOTACTL ; +extern struct req_msg_field RMF_OBD_QUOTACTL_POOL ;extern struct req_msg_field RMF_STRING ;extern struct req_msg_field RMF_SWAP_LAYOUTS ;extern struct req_msg_field RMF_MDS_HSM_PROGRESS ;
fs/lustre/include/lustre_swab.h +1 -1
@@ -60,7 +60,7 @@void lustre_swab_niobuf_remote (struct niobuf_remote *nbr);void lustre_swab_ost_lvb_v1 (struct ost_lvb_v1 *lvb);void lustre_swab_ost_lvb (struct ost_lvb *lvb); -void lustre_swab_obd_quotactl (struct obd_quotactl *q); +int lustre_swab_obd_quotactl (struct obd_quotactl *q,u32 len);void lustre_swab_lquota_lvb (struct lquota_lvb *lvb);void lustre_swab_generic_32s (u32 *val);void lustre_swab_mdt_body (struct mdt_body *b);
fs/lustre/include/obd_target.h new file +54 -0
54 lines not on this page · raw message
fs/lustre/llite/dir.c +25 -5
@@ -1083,15 +1083,19 @@ static int quotactl_ioctl(struct ll_sb_info *sbi, struct if_quotactl *qctl)case Q_SETQUOTA :case Q_SETINFO :case LUSTRE_Q_SETDEFAULT : +case LUSTRE_Q_SETQUOTAPOOL : +case LUSTRE_Q_SETINFOPOOL :if (!capable (CAP_SYS_ADMIN ))return -EPERM ;break ;case Q_GETQUOTA :case LUSTRE_Q_GETDEFAULT : +case LUSTRE_Q_GETQUOTAPOOL :if (check_owner (type, id) && !capable (CAP_SYS_ADMIN ))return -EPERM ;break ;case Q_GETINFO : +case LUSTRE_Q_GETINFOPOOL :break ;default :CERROR ("unsupported quotactl op: %#x\n" , cmd);
@@ -1101,7 +1105,8 @@ static int quotactl_ioctl(struct ll_sb_info *sbi, struct if_quotactl *qctl)if (valid !=QC_GENERAL ) {if (cmd ==Q_GETINFO ) qctl->qc_cmd =Q_GETOINFO ; -else if (cmd ==Q_GETQUOTA ) +else if (cmd ==Q_GETQUOTA || + cmd ==LUSTRE_Q_GETQUOTAPOOL ) qctl->qc_cmd =Q_GETOQUOTA ;else return -EINVAL ;
@@ -1134,8 +1139,12 @@ static int quotactl_ioctl(struct ll_sb_info *sbi, struct if_quotactl *qctl) qctl->qc_cmd = cmd; }else {struct obd_quotactl *oqctl; +int oqctl_len =sizeof (*oqctl); - oqctl =kzalloc (sizeof (*oqctl),GFP_NOFS ); +if (LUSTRE_Q_CMD_IS_POOL (cmd)) + oqctl_len +=LOV_MAXPOOLNAME +1 ; + + oqctl =kzalloc (oqctl_len,GFP_NOFS );if (!oqctl)return -ENOMEM ;
@@ -1148,7 +1157,7 @@ static int quotactl_ioctl(struct ll_sb_info *sbi, struct if_quotactl *qctl)/* If QIF_SPACE is not set, client should collect the * space usage from OSSs by itself */ -if (cmd ==Q_GETQUOTA && +if ((cmd ==Q_GETQUOTA || cmd ==LUSTRE_Q_GETQUOTAPOOL ) && !(oqctl->qc_dqblk.dqb_valid &QIF_SPACE ) && !oqctl->qc_dqblk.dqb_curspace) {struct obd_quotactl *oqctl_tmp;
@@ -1807,8 +1816,9 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) }case OBD_IOC_QUOTACTL : {struct if_quotactl *qctl; +int qctl_len =sizeof (*qctl) +LOV_MAXPOOLNAME +1 ; - qctl =kzalloc (sizeof (*qctl),GFP_NOFS ); + qctl =kzalloc (qctl_len,GFP_NOFS );if (!qctl)return -ENOMEM ;
@@ -1817,8 +1827,18 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) goto out_quotactl; } - rc =quotactl_ioctl (sbi, qctl); +if (LUSTRE_Q_CMD_IS_POOL (qctl->qc_cmd)) { +char __user *from = (char __user *)arg + + offsetof(typeof(*qctl), qc_poolname); +if (copy_from_user (qctl->qc_poolname, from, +LOV_MAXPOOLNAME +1 )) { + rc = -EFAULT ; + goto out_quotactl; + } + } + + rc =quotactl_ioctl (sbi, qctl);if (rc ==0 &©_to_user ((void __user *)arg, qctl,sizeof (*qctl))) rc = -EFAULT ;
fs/lustre/mdc/mdc_request.c +28 -12
@@ -2073,34 +2073,50 @@ static int mdc_quotactl(struct obd_device *unused, struct obd_export *exp,struct obd_quotactl *oqc;int rc; - req =ptlrpc_request_alloc_pack (class_exp2cliimp (exp), - &RQF_MDS_QUOTACTL ,LUSTRE_MDS_VERSION , -MDS_QUOTACTL ); + req =ptlrpc_request_alloc (class_exp2cliimp (exp), &RQF_MDS_QUOTACTL );if (!req)return -ENOMEM ; +if (LUSTRE_Q_CMD_IS_POOL (oqctl->qc_cmd)) +req_capsule_set_size (&req->rq_pill, + &RMF_OBD_QUOTACTL , +RCL_CLIENT , +sizeof (*oqc) +LOV_MAXPOOLNAME +1 ); + + rc =ptlrpc_request_pack (req,LUSTRE_MDS_VERSION , +MDS_QUOTACTL ); +if (rc) { +ptlrpc_request_free (req); +return rc; + } + oqc =req_capsule_client_get (&req->rq_pill, &RMF_OBD_QUOTACTL ); - *oqc = *oqctl; +QCTL_COPY (oqc, oqctl);ptlrpc_request_set_replen (req);ptlrpc_at_set_req_timeout (req); rc =ptlrpc_queue_wait (req); -if (rc) -CERROR ("ptlrpc_queue_wait failed, rc: %d\n" , rc); +if (rc) { +CERROR ("%s: ptlrpc_queue_wait failed: rc = %d\n" , + exp->exp_obd->obd_name, rc); + goto out; + }if (req->rq_repmsg) { oqc =req_capsule_server_get (&req->rq_pill, &RMF_OBD_QUOTACTL );if (oqc) { - *oqctl = *oqc; - }else if (!rc) { -CERROR ("Can't unpack obd_quotactl\n" ); - rc = -EPROTO ; +QCTL_COPY (oqctl, oqc); + goto out; } - }else if (!rc) { -CERROR ("Can't unpack obd_quotactl\n" ); + } + +if (!rc) { rc = -EPROTO ; +CERROR ("%s: cannot unpack obd_quotactl: rc = %d\n" , + exp->exp_obd->obd_name, rc); } +out:ptlrpc_req_finished (req);return rc;
fs/lustre/ptlrpc/layout.c +3 -3
@@ -923,9 +923,9 @@ struct req_msg_field RMF_MDT_BODY =EXPORT_SYMBOL (RMF_MDT_BODY );struct req_msg_field RMF_OBD_QUOTACTL = -DEFINE_MSGF ("obd_quotactl" ,0 , -sizeof (struct obd_quotactl ), - lustre_swab_obd_quotactl,NULL ); +DEFINE_MSGFL ("obd_quotactl" ,0 , +sizeof (struct obd_quotactl ), + lustre_swab_obd_quotactl,NULL );EXPORT_SYMBOL (RMF_OBD_QUOTACTL ); struct req_msg_field RMF_MDT_EPOCH =
fs/lustre/ptlrpc/pack_generic.c +6 -1
@@ -1893,14 +1893,19 @@ static void lustre_swab_obd_dqblk(struct obd_dqblk *b) BUILD_BUG_ON(offsetof(typeof(*b), dqb_padding) == 0); } -void lustre_swab_obd_quotactl (struct obd_quotactl *q) +int lustre_swab_obd_quotactl (struct obd_quotactl *q,u32 len) { +if (unlikely (len <=sizeof (struct obd_quotactl ))) +return -EOVERFLOW ; +__swab32s (&q->qc_cmd);__swab32s (&q->qc_type);__swab32s (&q->qc_id);__swab32s (&q->qc_stat);lustre_swab_obd_dqinfo (&q->qc_dqinfo);lustre_swab_obd_dqblk (&q->qc_dqblk); + +return len; }void lustre_swab_fid2path (struct getinfo_fid2path *gf)
include/uapi/linux/lustre/lustre_idl.h +13 -8
@@ -1363,18 +1363,23 @@ struct obd_quotactl {__u32 qc_stat;struct obd_dqinfo qc_dqinfo;struct obd_dqblk qc_dqblk; +char qc_poolname[0 ]; };#define Q_COPY (out, in, member) (out)->member = (in)->member -#define QCTL_COPY (out, in) \ -do { \ - Q_COPY(out, in, qc_cmd); \ - Q_COPY(out, in, qc_type); \ - Q_COPY(out, in, qc_id); \ - Q_COPY(out, in, qc_stat); \ - Q_COPY(out, in, qc_dqinfo); \ - Q_COPY(out, in, qc_dqblk); \ +#define QCTL_COPY (out, in) \ +do { \ + Q_COPY(out, in, qc_cmd); \ + Q_COPY(out, in, qc_type); \ + Q_COPY(out, in, qc_id); \ + Q_COPY(out, in, qc_stat); \ + Q_COPY(out, in, qc_dqinfo); \ + Q_COPY(out, in, qc_dqblk); \ + if (LUSTRE_Q_CMD_IS_POOL(in->qc_cmd)) \ + memcpy(out->qc_poolname, \ + in->qc_poolname, \ + LOV_MAXPOOLNAME + 1); \ } while (0)/* Data structures associated with the quota locks */
include/uapi/linux/lustre/lustre_user.h +11 -0
@@ -890,6 +890,10 @@ static inline void obd_uuid2fsname(char *buf, char *uuid, int buflen)#define LUSTRE_Q_FINVALIDATE 0x80000c/* deprecated as of 2.4 */ #define LUSTRE_Q_GETDEFAULT 0x80000d/* get default quota */ #define LUSTRE_Q_SETDEFAULT 0x80000e/* set default quota */ +#define LUSTRE_Q_GETQUOTAPOOL 0x80000f/* get user pool quota */ +#define LUSTRE_Q_SETQUOTAPOOL 0x800010/* set user pool quota */ +#define LUSTRE_Q_GETINFOPOOL 0x800011/* get pool quota info */ +#define LUSTRE_Q_SETINFOPOOL 0x800012/* set pool quota info */ /* In the current Lustre implementation, the grace time is either the time * or the timestamp to be used after some quota ID exceeds the soft limt,
@@ -911,6 +915,12 @@ static inline void obd_uuid2fsname(char *buf, char *uuid, int buflen)*/ #define LQUOTA_FLAG_DEFAULT 0x0001 +#define LUSTRE_Q_CMD_IS_POOL (cmd) \ + (cmd == LUSTRE_Q_GETQUOTAPOOL || \ + cmd == LUSTRE_Q_SETQUOTAPOOL || \ + cmd == LUSTRE_Q_SETINFOPOOL || \ + cmd == LUSTRE_Q_GETINFOPOOL) +#define ALLQUOTA 255/* set all quota */ static inline const char *qtype_name (int qtype)
@@ -1009,6 +1019,7 @@ struct if_quotactl {struct obd_dqblk qc_dqblk;char obd_type[16 ];struct obd_uuid obd_uuid; +char qc_poolname[0 ]; };/* swap layout flags */
Signature · 1 line
1.8.3.1
PATCH 18/45 lnet: libcfs: use BIT() macro where appropriate
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
When accessing a bit in a bitmap/mask/flags-word it can be more readable to use BIT(num) rather than "1 << num".
This patch makes that change to various places in libcfs.
WC-bug-id: https://jira.whamcloud.com/browse/LU-6142
Lustre-commit: 5bb99e46bc904 ("LU-6142 libcfs: use BIT() macro where appropriate")
Reviewed-on: https://review.whamcloud.com/38374
--- net/lnet/libcfs/debug.c | 2 +- net/lnet/libcfs/libcfs_string.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
net/lnet/libcfs/debug.c +1 -1
@@ -273,7 +273,7 @@ static const char *libcfs_debug_dbg2str(int debug) len =1 ; }else {/* space-separated tokens */ for (i =0 ; i <32 ; i++) { -if (!(mask & (1 << i))) +if ((mask &BIT (i)) ==0 )continue ; token =fn (i);
net/lnet/libcfs/libcfs_string.c +2 -2
@@ -88,9 +88,9 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit),if (debugstr &&strlen (debugstr) == len && !strncasecmp (str, debugstr, len)) {if (op =='-' ) - newmask &= ~(1 << i); + newmask &= ~BIT (i);else - newmask |= (1 << i); + newmask |=BIT (i); found =1 ;break ; }
Signature · 1 line
1.8.3.1
PATCH 19/45 lustre: llite: clean up pcc_layout_wait()
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
pcc_layout_wait() contains a pointless while loop (and an unnecessary blank line) and is a trivial function that is only called once.
So move the code into __pcc_layout_invalidate(), remove the while loop, and generally tidy up.
WC-bug-id: https://jira.whamcloud.com/browse/LU-10467
Lustre-commit: c5a311781fc60 ("LU-10467 llite: clean up pcc_layout_wait()")
Reviewed-on: https://review.whamcloud.com/38403
--- fs/lustre/llite/pcc.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)
fs/lustre/llite/pcc.c +7 -10
@@ -1968,20 +1968,17 @@ int pcc_fault(struct vm_area_struct *vma, struct vm_fault *vmf,return rc; } -static void pcc_layout_wait (struct pcc_inode *pcci) -{ -if (atomic_read (&pcci->pcci_active_ios) >0 ) -CDEBUG (D_CACHE ,"Waiting for IO completion: %d\n" , -atomic_read (&pcci->pcci_active_ios)); -wait_event_idle (pcci->pcci_waitq, -atomic_read (&pcci->pcci_active_ios) ==0 ); -} -static void __pcc_layout_invalidate (struct pcc_inode *pcci) { pcci->pcci_type =LU_PCC_NONE ;pcc_layout_gen_set (pcci,CL_LAYOUT_GEN_NONE ); -pcc_layout_wait (pcci); +if (atomic_read (&pcci->pcci_active_ios) ==0 ) +return ; + +CDEBUG (D_CACHE ,"Waiting for IO completion: %d\n" , +atomic_read (&pcci->pcci_active_ios)); +wait_event_idle (pcci->pcci_waitq, +atomic_read (&pcci->pcci_active_ios) ==0 ); }void pcc_layout_invalidate (struct inode *inode)
Signature · 1 line
1.8.3.1
PATCH 20/45 lustre: misc: declare static chars as const where possible.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
When a char * is statically initialized to a literal string, it is best if the char is declared 'const' to ensure that literal string can never be changed, and to allow it to be stored in read-only memory.
Unfortunately this is not possible for strings used as module parameters. Similarly osd_0copy_tag in osd-zfs cannot be made const, so that is change to a simple array (no pointer).
But in many other places it is possible, and it requires all ultimate users of the string to declare that they are working with const chars.
This patch changes a collection of static char*s to const, and in many cases arrays of char * to "const char * const", and then propagates the 'const' annotion as needed.
WC-bug-id: https://jira.whamcloud.com/browse/LU-6142
Lustre-commit: 257c0debc71ed ("LU-6142 misc: declare static chars as const where possible.")
Reviewed-on: https://review.whamcloud.com/38421
--- fs/lustre/include/lustre_import.h | 4 ++-- fs/lustre/obdclass/lprocfs_status.c | 2 +- include/linux/lnet/lib-lnet.h | 6 +++--- net/lnet/lnet/api-ni.c | 10 +++++----- net/lnet/lnet/config.c | 13 +++++++------ 5 files changed, 18 insertions(+), 17 deletions(-)
fs/lustre/include/lustre_import.h +2 -2
@@ -112,9 +112,9 @@ enum lustre_imp_state { };/** Returns test string representation of numeric import state @state */ -static inline char *ptlrpc_import_state_name (enum lustre_imp_state state) +static inline const char *ptlrpc_import_state_name (enum lustre_imp_state state) { -static char *import_state_names[] = { +static const char *const import_state_names[] = {"<UNKNOWN>" ,"CLOSED" ,"NEW" ,"DISCONN" ,"CONNECTING" ,"REPLAY" ,"REPLAY_LOCKS" ,"REPLAY_WAIT" ,"RECOVER" ,"FULL" ,"EVICTED" ,"IDLE" ,
fs/lustre/obdclass/lprocfs_status.c +1 -1
@@ -597,7 +597,7 @@ int lprocfs_rd_server_uuid(struct seq_file *m, void *data) {struct obd_device *obd = data;struct obd_import *imp; -char *imp_state_name =NULL ; +const char *imp_state_name =NULL ;int rc;LASSERT (obd);
include/linux/lnet/lib-lnet.h +3 -3
@@ -718,9 +718,9 @@ int lnet_push_target_post(struct lnet_ping_buffer *pbuf, struct lnet_handle_md *mdh);void lnet_peer_push_event (struct lnet_event *ev); -int lnet_parse_ip2nets (char **networksp,char *ip2nets); -int lnet_parse_routes (char *route_str,int *im_a_router); -int lnet_parse_networks (struct list_head *nilist,char *networks, +int lnet_parse_ip2nets (const char **networksp,const char *ip2nets); +int lnet_parse_routes (const char *route_str,int *im_a_router); +int lnet_parse_networks (struct list_head *nilist,const char *networks,bool use_tcp_bonding);bool lnet_net_unique (u32 net_id,struct list_head *nilist,struct lnet_net **net);
net/lnet/lnet/api-ni.c +5 -5
@@ -472,16 +472,16 @@ static int lnet_discover(struct lnet_process_id id, u32 force,return 0 ; } -static char * +static const char *lnet_get_routes (void ) {return routes; } -static char * +static const char *lnet_get_networks (void ) { -char *nets; +const char *nets;int rc;if (*networks && *ip2nets) {
@@ -3101,7 +3101,7 @@ static int lnet_handle_legacy_ip2nets(char *ip2nets, struct lnet_ioctl_config_lnd_tunables *tun) {struct lnet_net *net; -char *nets; +const char *nets;int rc;LIST_HEAD (net_head);
@@ -3286,7 +3286,7 @@ int lnet_dyn_del_ni(struct lnet_ioctl_config_ni *conf)LIST_HEAD (net_head);int rc;struct lnet_ioctl_config_lnd_tunables tun; -char *nets = conf->cfg_config_u.cfg_net.net_intf; +const char *nets = conf->cfg_config_u.cfg_net.net_intf;/* Create a net/ni structures for the network string */ rc =lnet_parse_networks (&net_head, nets, use_tcp_bonding);
net/lnet/lnet/config.c +7 -6
@@ -568,7 +568,7 @@ struct lnet_ni ** nilist. */ int -lnet_parse_networks (struct list_head *netlist,char *networks, +lnet_parse_networks (struct list_head *netlist,const char *networks,bool use_tcp_bonding) {struct cfs_expr_list *net_el =NULL ;
@@ -887,10 +887,10 @@ struct lnet_ni * }static int -lnet_str2tbs_sep (struct list_head *tbs,char *str) +lnet_str2tbs_sep (struct list_head *tbs,const char *str) {LIST_HEAD (pending); -char *sep; +const char *sep;int nob;int i;struct lnet_text_buf *ltb;
@@ -1250,7 +1250,7 @@ struct lnet_ni * }int -lnet_parse_routes (char *routes,int *im_a_router) +lnet_parse_routes (const char *routes,int *im_a_router) {LIST_HEAD (tbs);int rc =0 ;
@@ -1439,7 +1439,8 @@ struct lnet_ni * }static int -lnet_match_networks (char **networksp,char *ip2nets,u32 *ipaddrs,int nip) +lnet_match_networks (const char **networksp,const char *ip2nets, +u32 *ipaddrs,int nip) {static char networks[LNET_SINGLE_TEXTBUF_NOB ];static char source[LNET_SINGLE_TEXTBUF_NOB ];
@@ -1620,7 +1621,7 @@ int lnet_inet_enumerate(struct lnet_inetdev **dev_list, struct net *ns)EXPORT_SYMBOL (lnet_inet_enumerate);int -lnet_parse_ip2nets (char **networksp,char *ip2nets) +lnet_parse_ip2nets (const char **networksp,const char *ip2nets) {struct lnet_inetdev *ifaces =NULL ;u32 *ipaddrs =NULL ;
Signature · 1 line
1.8.3.1
PATCH 21/45 lustre: llite: fix to make jobstats work for async ra
In reply to James SimmonsFrom: Wang Shilong <[email protected]>
We use workqueue to triger async readahead, this will make the problem that read jobstats are not accounted by real IO, we could fix this by overwritting jobid which inited in vvp_io_init().
WC-bug-id: https://jira.whamcloud.com/browse/LU-13490
Lustre-commit: da8972322134a ("LU-13490 lustre: fix to make jobstats work for async ra")
Reviewed-on: https://review.whamcloud.com/38426
--- fs/lustre/llite/llite_internal.h | 1 + fs/lustre/llite/rw.c | 8 ++++++++ 2 files changed, 9 insertions(+)
fs/lustre/llite/llite_internal.h +1 -0
@@ -780,6 +780,7 @@ struct ll_readahead_work {/* async worker to handler read */ struct work_struct lrw_readahead_work; +char lrw_jobid[LUSTRE_JOBID_SIZE ]; };extern struct kmem_cache *ll_file_data_slab;
fs/lustre/llite/rw.c +8 -0
@@ -629,6 +629,12 @@ static void ll_readahead_handle_work(struct work_struct *wq)if (rc) goto out_put_env; +/* overwrite jobid inited in vvp_io_init() */ +if (strncmp (ll_i2info (inode)->lli_jobid, work->lrw_jobid, +sizeof (work->lrw_jobid))) +memcpy (ll_i2info (inode)->lli_jobid, work->lrw_jobid, +sizeof (work->lrw_jobid)); +vvp_env_io (env)->vui_fd = fd; io->ci_state =CIS_LOCKED ; io->ci_async_readahead = true;
@@ -1582,6 +1588,8 @@ static int kickoff_async_readahead(struct file *file, unsigned long pages) ras->ras_next_readahead_idx = end_idx +1 ; ras->ras_async_last_readpage_idx = start_idx;spin_unlock (&ras->ras_lock); +memcpy (lrw->lrw_jobid,ll_i2info (inode)->lli_jobid, +sizeof (lrw->lrw_jobid));ll_readahead_work_add (inode, lrw); }else {return -ENOMEM ;
Signature · 1 line
1.8.3.1
PATCH 22/45 lustre: llite: verify truncated xattr is handled
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
Verify that a truncated trusted.lov xattr is handled properly, for both plain and PFL layouts.
Add a test case that verifies this is fixed for both layout types.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13168
Lustre-commit: cb74546354201 ("LU-13168 tests: verify truncated xattr is handled")
Reviewed-on: https://review.whamcloud.com/38434
--- fs/lustre/llite/xattr.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
fs/lustre/llite/xattr.c +12 -2
@@ -190,7 +190,8 @@ static int get_hsm_state(struct inode *inode, u32 *hus_states)return rc; } -static int ll_adjust_lum (struct inode *inode,struct lov_user_md *lump) +static int ll_adjust_lum (struct inode *inode,struct lov_user_md *lump, +size_t size) {struct lov_comp_md_v1 *comp_v1 = (struct lov_comp_md_v1 *)lump;struct lov_user_md *v1 = lump;
@@ -205,7 +206,12 @@ static int ll_adjust_lum(struct inode *inode, struct lov_user_md *lump)return 0 ;if (lump->lmm_magic ==LOV_USER_MAGIC_COMP_V1 ) { +if (size <sizeof (*comp_v1)) +return -ERANGE ; + entry_count = comp_v1->lcm_entry_count; + if (size < offsetof(typeof(*comp_v1), lcm_entries[entry_count])) +return -ERANGE ; is_composite = true; }
@@ -213,6 +219,10 @@ static int ll_adjust_lum(struct inode *inode, struct lov_user_md *lump)if (lump->lmm_magic ==LOV_USER_MAGIC_COMP_V1 ) {void *ptr = comp_v1; +if (comp_v1->lcm_entries[i].lcme_offset +sizeof (*v1) > + size) +return -ERANGE ; + ptr += comp_v1->lcm_entries[i].lcme_offset; v1 = (struct lov_user_md *)ptr; }
@@ -265,7 +275,7 @@ static int ll_setstripe_ea(struct dentry *dentry, struct lov_user_md *lump,return -ERANGE ; } - rc =ll_adjust_lum (inode, lump); + rc =ll_adjust_lum (inode, lump, size);if (rc)return rc;
Signature · 1 line
1.8.3.1
PATCH 23/45 lustre: obd: fix printing of client connection UUID
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
The client connection UUID sent to the servers (ASCII format) was being truncated to only 16 bytes in size, like '595f3c6a-20ae-4' instead of a full UUID like '18ae0f9a-4b09-4599-8ced-0f2126eab425'.
This was caused by using UUID_SIZE to limit the size of the "%pU" string printed to avoid overflowing the target buffer, but in fact UUID_SIZE is the size of the binary uuid_t (16 bytes) instead of the size of struct obd_uuid (40 bytes) where the ASCII version of the UUID is stored.
Fix this to use sizeof(target) rather than an external constant, which is exactly why sizeof(target) should always be used. The usage in osd_scrub.c is not actually broken, but it is still better to use sizeof(target) to avoid future inconsistencies.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13499
Lustre-commit: 9abdf96e56e23 ("LU-13499 obd: fix printing of client connection UUID")
Reviewed-on: https://review.whamcloud.com/38443
--- fs/lustre/llite/llite_lib.c | 2 +- fs/lustre/obdclass/obd_mount.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
fs/lustre/llite/llite_lib.c +1 -1
@@ -1046,7 +1046,7 @@ int ll_fill_super(struct super_block *sb)/* UUID handling */ generate_random_uuid (uuid.b); -snprintf (sbi->ll_sb_uuid.uuid,UUID_SIZE ,"%pU" , uuid.b); +snprintf (sbi->ll_sb_uuid.uuid,sizeof (sbi->ll_sb_uuid),"%pU" , uuid.b);CDEBUG (D_CONFIG ,"llite sb uuid: %s\n" , sbi->ll_sb_uuid.uuid);
fs/lustre/obdclass/obd_mount.c +1 -1
@@ -339,7 +339,7 @@ int lustre_start_mgc(struct super_block *sb) }generate_random_uuid (uuidc.b); -snprintf (uuid->uuid,UUID_SIZE ,"%pU" , uuidc.b); +snprintf (uuid->uuid,sizeof (*uuid),"%pU" , uuidc.b);/* Start the MGC */ rc = lustre_start_simple(mgcname, LUSTRE_MGC_NAME,
Signature · 1 line
1.8.3.1
PATCH 24/45 lnet: Add MD options for response tracking
In reply to James SimmonsFrom: Chris Horn <[email protected]>
These MD options will allow response tracking on a particular MD to override the behavior specified by the lnet_response_tracking parameter.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13502
Lustre-commit: 448834b8e575f ("LU-13502 lnet: Add MD options for response tracking")
Reviewed-on: https://review.whamcloud.com/38450
--- include/uapi/linux/lnet/lnet-types.h | 8 ++++++++ 1 file changed, 8 insertions(+)
include/uapi/linux/lnet/lnet-types.h +8 -0
@@ -483,6 +483,10 @@ struct lnet_md {* struct bio_vec. * - LNET_MD_MAX_SIZE: The max_size field is valid. * - LNET_MD_BULK_HANDLE: The bulk_handle field is valid. +* - LNET_MD_TRACK_RESPONSE: Enable response tracking on this MD +* regardless of the value of the lnet_response_tracking param. +* - LNET_MD_NO_TRACK_RESPONSE: Disable response tracking on this MD +* regardless of the value of the lnet_response_tracking param. * * Note: * - LNET_MD_KIOV allows for a scatter/gather capability for memory
@@ -545,6 +549,10 @@ struct lnet_md {#define LNET_MD_KIOV (1 << 8)/** See lnet_md::options. */ #define LNET_MD_BULK_HANDLE (1 << 9) +/** See struct lnet_md::options. */ +#define LNET_MD_TRACK_RESPONSE (1 << 10) +/** See struct lnet_md::options. */ +#define LNET_MD_NO_TRACK_RESPONSE (1 << 11)/* For compatibility with Cray Portals */ #define LNET_MD_PHYS 0
Signature · 1 line
1.8.3.1
PATCH 25/45 lustre: Send file creation time to clients
In reply to James SimmonsFrom: Qian Yingjin <[email protected]>
Both ext4 and ZFS allow storing the file creation time in the on-disk inode, and the new statx() API allows returning it to userspace, but as yet we do not have any mechanism to send it from the servers to the client.
Add fields into struct mdt_body and an OBD_MD_FLBTIME flag to allow it to be requested and returned directly from MDTs. It does not need to get it from the OSTs or in the LVB since those objects are precreated and their creation time (birth time) is not accurate.
WC-bug-id: https://jira.whamcloud.com/browse/LU-11971
Lustre-commit: 186b97e68abbc ("LU-11971 utils: Send file creation time to clients")
Reviewed-on: https://review.whamcloud.com/36507
--- fs/lustre/include/lu_object.h | 2 ++ fs/lustre/llite/dir.c | 3 ++- fs/lustre/ptlrpc/pack_generic.c | 2 +- fs/lustre/ptlrpc/wiretest.c | 8 ++++---- include/uapi/linux/lustre/lustre_idl.h | 6 ++++-- include/uapi/linux/lustre/lustre_user.h | 1 + 6 files changed, 14 insertions(+), 8 deletions(-)
fs/lustre/include/lu_object.h +2 -0
@@ -399,6 +399,8 @@ struct lu_attr {s64 la_atime;/** change time in seconds since Epoch */ s64 la_ctime; +/** create time in seconds since Epoch */ +s64 la_btime;/** 512-byte blocks allocated to object */ u64 la_blocks;/** permission bits and file type */
fs/lustre/llite/dir.c +2 -1
@@ -1758,11 +1758,12 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) stx.stx_atime.tv_sec = body->mbo_atime; stx.stx_ctime.tv_sec = body->mbo_ctime; stx.stx_mtime.tv_sec = body->mbo_mtime; + stx.stx_btime.tv_sec = body->mbo_btime; stx.stx_rdev_major =MAJOR (body->mbo_rdev); stx.stx_rdev_minor =MINOR (body->mbo_rdev); stx.stx_dev_major =MAJOR (inode->i_sb->s_dev); stx.stx_dev_minor =MINOR (inode->i_sb->s_dev); - stx.stx_mask |=STATX_BASIC_STATS ; + stx.stx_mask |=STATX_BASIC_STATS |STATX_BTIME ;/* * For a striped directory, the size and blocks returned
fs/lustre/ptlrpc/pack_generic.c +1 -1
@@ -1805,7 +1805,7 @@ void lustre_swab_mdt_body(struct mdt_body *b)__swab32s (&b->mbo_projid);__swab64s (&b->mbo_dom_size);__swab64s (&b->mbo_dom_blocks); - BUILD_BUG_ON(offsetof(typeof(*b), mbo_padding_8) == 0); +__swab64s (&b->mbo_btime); BUILD_BUG_ON(offsetof(typeof(*b), mbo_padding_9) == 0); BUILD_BUG_ON(offsetof(typeof(*b), mbo_padding_10) == 0); }
fs/lustre/ptlrpc/wiretest.c +4 -4
@@ -2218,10 +2218,10 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct mdt_body, mbo_dom_blocks));LASSERTF ((int )sizeof (((struct mdt_body *)0 )->mbo_dom_blocks) ==8 ,"found %lld\n" , (long long )(int )sizeof (((struct mdt_body *)0 )->mbo_dom_blocks)); -LASSERTF ((int )offsetof(struct mdt_body , mbo_padding_8) ==192 ,"found %lld\n" , - (long long )(int )offsetof(struct mdt_body , mbo_padding_8)); -LASSERTF ((int )sizeof (((struct mdt_body *)0 )->mbo_padding_8) ==8 ,"found %lld\n" , - (long long )(int )sizeof (((struct mdt_body *)0 )->mbo_padding_8)); +LASSERTF ((int )offsetof(struct mdt_body , mbo_btime) ==192 ,"found %lld\n" , + (long long )(int )offsetof(struct mdt_body , mbo_btime)); +LASSERTF ((int )sizeof (((struct mdt_body *)0 )->mbo_btime) ==8 ,"found %lld\n" , + (long long )(int )sizeof (((struct mdt_body *)0 )->mbo_btime));LASSERTF ((int )offsetof(struct mdt_body , mbo_padding_9) ==200 ,"found %lld\n" , (long long )(int )offsetof(struct mdt_body , mbo_padding_9)); LASSERTF((int)sizeof(((struct mdt_body *)0)->mbo_padding_9) == 8, "found %lld\n",
include/uapi/linux/lustre/lustre_idl.h +4 -2
@@ -1213,6 +1213,8 @@ static inline __u32 lov_mds_md_size(__u16 stripes, __u32 lmm_magic)#define OBD_MD_FLLAZYSIZE (0x0400000000000000ULL)/* Lazy size */ #define OBD_MD_FLLAZYBLOCKS (0x0800000000000000ULL)/* Lazy blocks */ +#define OBD_MD_FLBTIME (0x1000000000000000ULL)/* birth time */ +#define OBD_MD_FLALLQUOTA (OBD_MD_FLUSRQUOTA | \ OBD_MD_FLGRPQUOTA | \ OBD_MD_FLPRJQUOTA)
@@ -1222,7 +1224,7 @@ static inline __u32 lov_mds_md_size(__u16 stripes, __u32 lmm_magic)OBD_MD_FLMODE |OBD_MD_FLTYPE |OBD_MD_FLUID | \OBD_MD_FLGID |OBD_MD_FLFLAGS |OBD_MD_FLNLINK | \OBD_MD_FLPARENT |OBD_MD_FLRDEV |OBD_MD_FLGROUP | \ - OBD_MD_FLPROJID) + OBD_MD_FLPROJID | OBD_MD_FLBTIME) #define OBD_MD_FLXATTRALL (OBD_MD_FLXATTR | OBD_MD_FLXATTRLS)
@@ -1589,7 +1591,7 @@ struct mdt_body {__u32 mbo_projid;/* also fix lustre_swab_mdt_body */ __u64 mbo_dom_size;/* size of DOM component */ __u64 mbo_dom_blocks;/* blocks consumed by DOM component */ -__u64 mbo_padding_8;/* also fix lustre_swab_mdt_body */ +__u64 mbo_btime;__u64 mbo_padding_9;__u64 mbo_padding_10; };/* 216 */
include/uapi/linux/lustre/lustre_user.h +1 -0
@@ -1064,6 +1064,7 @@ enum la_valid {LA_LAYOUT_VERSION =1 <<16 ,LA_LSIZE =1 <<17 ,LA_LBLOCKS =1 <<18 , + LA_BTIME = 1 << 19, /* 0x8000 *//** * Attributes must be transmitted to OST objects */
Signature · 1 line
1.8.3.1
PATCH 26/45 lnet: stop using struct timeval
In reply to James SimmonsThe struct timeval is not 2038 safe so the Linux kernel is moving away from its use. The use of rpe_stamp hasn't been used since Lustre 2.2 so remove the userland use of this field. This frees use to change rpe_stamp to an equivalent struct timespec64 for future use. Greatly simplify lnet_sock_[read|write] by using jiffies values of sk_sndtimeo, sk_rcvtimeo cached in struct sock.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13344
Lustre-commit: 5e4c658c4fea4 ("LU-13344 lnet: stop using struct timeval")
Reviewed-on: https://review.whamcloud.com/38105
--- include/uapi/linux/lnet/lnetst.h | 8 +++++++- include/uapi/linux/lustre/lustre_user.h | 2 +- net/lnet/lnet/lib-socket.c | 34 ++++++++++----------------------- net/lnet/selftest/conctl.c | 1 - net/lnet/selftest/conrpc.c | 6 +++--- net/lnet/selftest/conrpc.h | 1 - net/lnet/selftest/console.h | 1 - 7 files changed, 21 insertions(+), 32 deletions(-)
include/uapi/linux/lnet/lnetst.h +7 -1
@@ -149,7 +149,13 @@ struct list_head {struct lstcon_rpc_ent {struct list_head rpe_link;/* link chain */ struct lnet_process_id rpe_peer;/* peer's id */ -struct timeval rpe_stamp;/* time stamp of RPC */ +/* This has not been used since Lustre 2.2 so its safe to use. +* Update to allow future use of timespec64 +*/ +struct { +__s64 tv_sec; +__s64 tv_nsec; + } rpe_stamp;/* time stamp of RPC */ int rpe_state;/* peer's state */ int rpe_rpc_errno;/* RPC errno */
include/uapi/linux/lustre/lustre_user.h +1 -1
@@ -961,7 +961,7 @@ struct identity_downcall_data {struct sepol_downcall_data {__u32 sdd_magic; -__kernel_time_t sdd_sepol_mtime; +__s64 sdd_sepol_mtime;__u16 sdd_sepol_len;char sdd_sepol[0 ]; };
net/lnet/lnet/lib-socket.c +10 -24
@@ -50,8 +50,6 @@int rc;long jiffies_left = timeout *HZ ;unsigned long then; -struct timeval tv; -struct __kernel_sock_timeval ktv;struct kvec iov = { .iov_base = buffer, .iov_len = nob
@@ -67,17 +65,12 @@for (;;) { msg.msg_flags = !timeout ?MSG_DONTWAIT :0 ;if (timeout) { +struct sock *sk = sock->sk; +/* Set send timeout to remaining time */ -jiffies_to_timeval (jiffies_left, &tv); - ktv.tv_sec = tv.tv_sec; - ktv.tv_usec = tv.tv_usec; - rc =kernel_setsockopt (sock,SOL_SOCKET ,SO_SNDTIMEO_NEW , - (char *)&ktv,sizeof (ktv)); -if (rc) { -CERROR ("Can't set socket send timeout %ld.%06d: %d\n" , - (long )tv.tv_sec, (int )tv.tv_usec, rc); -return rc; - } +lock_sock (sk); + sk->sk_sndtimeo = jiffies_left; +release_sock (sk); } then = jiffies;
@@ -108,8 +101,6 @@int rc;long jiffies_left = timeout *HZ ;unsigned long then; -struct timeval tv; -struct __kernel_sock_timeval ktv;struct kvec iov = { .iov_base = buffer, .iov_len = nob
@@ -124,17 +115,12 @@iov_iter_kvec (&msg.msg_iter,READ , &iov,1 , nob);for (;;) { +struct sock *sk = sock->sk; +/* Set receive timeout to remaining time */ -jiffies_to_timeval (jiffies_left, &tv); - ktv.tv_sec = tv.tv_sec; - ktv.tv_usec = tv.tv_usec; - rc =kernel_setsockopt (sock,SOL_SOCKET ,SO_RCVTIMEO_NEW , - (char *)&ktv,sizeof (ktv)); -if (rc) { -CERROR ("Can't set socket recv timeout %ld.%06d: %d\n" , - (long )tv.tv_sec, (int )tv.tv_usec, rc); -return rc; - } +lock_sock (sk); + sk->sk_rcvtimeo = jiffies_left; +release_sock (sk); then = jiffies; rc =sock_recvmsg (sock, &msg,0 );
net/lnet/selftest/conctl.c +0 -1
@@ -38,7 +38,6 @@*/ #include <linux/lnet/lib-lnet.h> -#include <uapi/linux/lnet/lnetst.h> #include "console.h" static int
net/lnet/selftest/conrpc.c +3 -3
@@ -474,7 +474,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *,struct lstcon_rpc *crpc;struct srpc_msg *msg;struct lstcon_node *nd; -struct timeval tv; +struct timespec64 ts;int error;s64 dur;
@@ -501,11 +501,11 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, dur = crpc->crp_stamp_ns - console_session.ses_id.ses_stamp *NSEC_PER_MSEC ; - tv =ns_to_timeval (dur); + ts =ns_to_timespec64 (dur);if (copy_to_user (&ent->rpe_peer, &nd->nd_id,sizeof (struct lnet_process_id )) || -copy_to_user (&ent->rpe_stamp, &tv,sizeof (tv)) || +copy_to_user (&ent->rpe_stamp, &ts,sizeof (ts)) ||copy_to_user (&ent->rpe_state, &nd->nd_state,sizeof (nd->nd_state)) || copy_to_user(&ent->rpe_rpc_errno, &error,
net/lnet/selftest/conrpc.h +0 -1
@@ -41,7 +41,6 @@#define __LST_CONRPC_H__#include <linux/lnet/lib-types.h> -#include <uapi/linux/lnet/lnetst.h> #include "rpc.h" #include "selftest.h"
net/lnet/selftest/console.h +0 -1
@@ -41,7 +41,6 @@#define __LST_CONSOLE_H__#include <linux/lnet/lib-types.h> -#include <uapi/linux/lnet/lnetst.h> #include "selftest.h" #include "conrpc.h"
Signature · 1 line
1.8.3.1
PATCH 27/45 lustre: ptlrpc: connect to MDT stucks
In reply to James SimmonsFrom: Andriy Skulysh <[email protected]>
mdc_ioc_hsm_ct_register() is executed onIMP_EVENT_ACTIVE event.
If HSM_CT_REGISTER request expires import becomes
disconnected but reconnect can't start because
previous connect waits for completion ofHSM_CT_REGISTER request.
Mark HSM_CT_REGISTER request as no_resend.
Cray-bug-id: LUS-8409
WC-bug-id: https://jira.whamcloud.com/browse/LU-13455
Lustre-commit: 3d58403e62b7b ("LU-13455 ptlrpc: connect to MDT stucks")
Reviewed-on: https://review.whamcloud.com/38243
--- fs/lustre/mdc/mdc_request.c | 1 + 1 file changed, 1 insertion(+)
fs/lustre/mdc/mdc_request.c +1 -0
@@ -1845,6 +1845,7 @@ static int mdc_ioc_hsm_ct_register(struct obd_import *imp, u32 archive_count, *archive_array = archive_count;ptlrpc_request_set_replen (req); + req->rq_no_resend =1 ; rc =mdc_queue_wait (req); out:
Signature · 1 line
1.8.3.1
PATCH 28/45 lnet: restrict gateway selection
In reply to James SimmonsFrom: Amir Shehata <[email protected]>
This patch fixes a conflict between LU-13026 lnet: MR selection of gateway ni and
LU-12919 lnet: Fix source specified route selection
LU-12919 patch relied on lnet_find_best_lpni_on_net() to restrict lpni selection on a specific network. However, it is necessary to allow lpni selection on any network if the src net is not specified. LU-13026 removed the restriction imposed by lnet_find_best_lpni_on_net(), which broke the restriction on the source net selection in some configuration setups.
This patch renames lnet_find_best_lpni_on_net() to lnet_find_best_lpni(). If passed a peer_net it will find the best lpni on the peer_net or return NULL if none is available. If passed 'any' net, then it'll find the best reachable lpni on any available net.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13461
Lustre-commit: ceb92c5512bad ("LU-13461 lnet: restrict gateway selection")
Reviewed-on: https://review.whamcloud.com/38298
--- net/lnet/lnet/lib-move.c | 91 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 64 insertions(+), 27 deletions(-)
net/lnet/lnet/lib-move.c +64 -27
@@ -1143,6 +1143,7 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,static struct lnet_peer_ni *lnet_select_peer_ni (struct lnet_ni *best_ni,lnet_nid_t dst_nid,struct lnet_peer *peer, +struct lnet_peer_ni *best_lpni,struct lnet_peer_net *peer_net) {/* Look at the peer NIs for the destination peer that connect
@@ -1153,11 +1154,12 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,* credits are equal, we round-robin over the peer_ni. */ struct lnet_peer_ni *lpni =NULL ; -struct lnet_peer_ni *best_lpni =NULL ; -int best_lpni_credits =INT_MIN ; +int best_lpni_credits = (best_lpni) ? best_lpni->lpni_txcredits : +INT_MIN ; +int best_lpni_healthv = (best_lpni) ? +atomic_read (&best_lpni->lpni_healthv) :0 ;bool preferred = false;bool ni_is_pref; -int best_lpni_healthv =0 ;int lpni_healthv;while ((lpni =lnet_get_next_peer_ni_locked (peer, peer_net, lpni))) {
@@ -1231,19 +1233,43 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,return best_lpni; } -/* Prerequisite: the best_ni should already be set in the sd */ +/* Prerequisite: the best_ni should already be set in the sd +* Find the best lpni. +* If the net id is provided then restrict lpni selection on +* that particular net. +* Otherwise find any reachable lpni. When dealing with an MR +* gateway and it has multiple lpnis which we can use +* we want to select the best one from the list of reachable +* ones. +*/ static inline struct lnet_peer_ni * -lnet_find_best_lpni_on_net (struct lnet_ni *lni,lnet_nid_t dst_nid, -struct lnet_peer *peer,u32 net_id) +lnet_find_best_lpni (struct lnet_ni *lni,lnet_nid_t dst_nid, +struct lnet_peer *peer,u32 net_id) {struct lnet_peer_net *peer_net; +u32 any_net =LNET_NIDNET (LNET_NID_ANY ); -/* The gateway is Multi-Rail capable so now we must select the -* proper peer_ni -*/ +/* find the best_lpni on any local network */ +if (net_id == any_net) { +struct lnet_peer_ni *best_lpni =NULL ; +struct lnet_peer_net *lpn; + +list_for_each_entry (lpn, &peer->lp_peer_nets, lpn_peer_nets) { +/* no net specified find any reachable peer ni */ +if (!lnet_islocalnet_locked (lpn->lpn_net_id)) +continue ; + best_lpni =lnet_select_peer_ni (lni, dst_nid, peer, + best_lpni, lpn); + } + +return best_lpni; + } +/* restrict on the specified net */ peer_net =lnet_peer_get_net_locked (peer, net_id); +if (peer_net) +return lnet_select_peer_ni (lni, dst_nid, peer,NULL , peer_net); -return lnet_select_peer_ni (lni, dst_nid, peer, peer_net); +return NULL ; }/* Compare route priorities and hop counts */
@@ -1279,6 +1305,9 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,struct lnet_route *route;int rc; +CDEBUG (D_NET ,"Looking up a route to %s, from %s\n" , +libcfs_net2str (rnet->lrn_net),libcfs_net2str (src_net)); + best_route =NULL ; last_route =NULL ;list_for_each_entry (route, &rnet->lrn_routes, lr_list) {
@@ -1290,15 +1319,16 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,* the best interface available. */ if (!best_route) { - lpni =lnet_find_best_lpni_on_net (NULL ,LNET_NID_ANY , - route->lr_gateway, - src_net); + lpni =lnet_find_best_lpni (NULL ,LNET_NID_ANY , + route->lr_gateway, + src_net);if (lpni) { best_route = route; last_route = route; best_gw_ni = lpni; }else { -CERROR ("Gateway %s does not have a peer NI on net %s\n" , +CDEBUG (D_NET , +"Gateway %s does not have a peer NI on net %s\n" ,libcfs_nid2str (route->lr_gateway->lp_primary_nid),libcfs_net2str (src_net)); }
@@ -1313,11 +1343,13 @@ void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats,if (rc ==-1 )continue ; - lpni =lnet_find_best_lpni_on_net (NULL ,LNET_NID_ANY , - route->lr_gateway, - src_net); + lpni =lnet_find_best_lpni (NULL ,LNET_NID_ANY , + route->lr_gateway, + src_net); +/* restrict the lpni on the src_net if specified */ if (!lpni) { -CERROR ("Gateway %s does not have a peer NI on net %s\n" , +CDEBUG (D_NET , +"Gateway %s does not have a peer NI on net %s\n" ,libcfs_nid2str (route->lr_gateway->lp_primary_nid),libcfs_net2str (src_net));continue ;
@@ -1794,7 +1826,12 @@ struct lnet_ni *struct lnet_route *last_route =NULL ;struct lnet_peer_ni *lpni =NULL ;struct lnet_peer_ni *gwni =NULL ; -lnet_nid_t src_nid = sd->sd_src_nid; +lnet_nid_t src_nid = (sd->sd_src_nid !=LNET_NID_ANY ) ? sd->sd_src_nid : + sd->sd_best_ni ? sd->sd_best_ni->ni_nid : +LNET_NID_ANY ; + +CDEBUG (D_NET ,"using src nid %s for route restriction\n" , +libcfs_nid2str (src_nid));/* If a router nid was specified then we are replying to a GET or * sending an ACK. In this case we use the gateway associated with the
@@ -1842,12 +1879,12 @@ struct lnet_ni *return -EHOSTUNREACH ; } - sd->sd_best_lpni =lnet_find_best_lpni_on_net (sd->sd_best_ni, - sd->sd_dst_nid, - lp, - best_lpn->lpn_net_id); + sd->sd_best_lpni =lnet_find_best_lpni (sd->sd_best_ni, + sd->sd_dst_nid, + lp, + best_lpn->lpn_net_id);if (!sd->sd_best_lpni) { -CERROR ("peer %s down\n" , +CERROR ("peer %s is unreachable\n" ,libcfs_nid2str (sd->sd_dst_nid));return -EHOSTUNREACH ; }
@@ -2176,9 +2213,9 @@ struct lnet_ni * lnet_msg_discovery(sd->sd_msg));if (sd->sd_best_ni) { sd->sd_best_lpni = -lnet_find_best_lpni_on_net (sd->sd_best_ni, sd->sd_dst_nid, - sd->sd_peer, - sd->sd_best_ni->ni_net->net_id); +lnet_find_best_lpni (sd->sd_best_ni, sd->sd_dst_nid, + sd->sd_peer, + sd->sd_best_ni->ni_net->net_id);/* if we're successful in selecting a peer_ni on the local * network, then send to it. Otherwise fall through and
Signature · 1 line
1.8.3.1
PATCH 29/45 lustre: llite: restore ll_dcompare()
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
Revert "LU-9868 llite: Get rid of ll_dcompare" as it is causing failures in conf-sanity test_32* due to errors with sub-mounted ext2 filesystem dentries being invalidated.
This reverts commit 8f0a4c7b907a27e77ea0dd80198478747c87bfba.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13486
Lustre-commit: 00fa5e8ef4559 ("LU-13486 llite: restore ll_dcompare()")
Reviewed-on: https://review.whamcloud.com/38498
--- fs/lustre/llite/dcache.c | 53 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 43 insertions(+), 10 deletions(-)
fs/lustre/llite/dcache.c +43 -10
@@ -61,6 +61,48 @@ static void ll_release(struct dentry *de)call_rcu (&lld->lld_rcu_head, free_dentry_data); } +/* Compare if two dentries are the same. Don't match if the existing dentry +* is marked invalid. Returns 1 if different, 0 if the same. +* +* This avoids a race where ll_lookup_it() instantiates a dentry, but we get +* an AST before calling d_revalidate_it(). The dentry still exists (marked +* INVALID) so d_lookup() matches it, but we have no lock on it (so +* lock_match() fails) and we spin around real_lookup(). +* +* This race doesn't apply to lookups in d_alloc_parallel(), and for +* those we want to ensure that only one dentry with a given name is +* in ll_lookup_nd() at a time. So allow invalid dentries to match +* while d_in_lookup(). We will be called again when the lookup +* completes, and can give a different answer then. +*/ +static int ll_dcompare (const struct dentry *dentry, +unsigned int len,const char *str, +const struct qstr *name) +{ +if (len != name->len) +return 1 ; + +if (memcmp (str, name->name, len)) +return 1 ; + +CDEBUG (D_DENTRY ,"found name %.*s(%p) flags %#x refc %d\n" , + name->len, name->name, dentry, dentry->d_flags, +d_count (dentry)); + +/* mountpoint is always valid */ +if (d_mountpoint (dentry)) +return 0 ; + +/* ensure exclusion against parallel lookup of the same name */ +if (d_in_lookup ((struct dentry *)dentry)) +return 0 ; + +if (d_lustre_invalid (dentry)) +return 1 ; + +return 0 ; +} +/** * Called when last reference to a dentry is dropped and dcache wants to know * whether or not it should cache it:
@@ -202,16 +244,6 @@ static int ll_revalidate_dentry(struct dentry *dentry, {struct inode *dir =d_inode (dentry->d_parent); -CDEBUG (D_VFSTRACE ,"VFS Op:name=%s, flags=%u\n" , - dentry->d_name.name, lookup_flags); - -/* mountpoint is always valid */ -if (d_mountpoint (dentry)) -return 1 ; - -if (d_lustre_invalid (dentry)) -return 0 ; -/* If this is intermediate component path lookup and we were able to get * to this dentry, then its lock has not been revoked and the * path component is valid.
@@ -258,4 +290,5 @@ static int ll_revalidate_nd(struct dentry *dentry, unsigned int flags) .d_revalidate = ll_revalidate_nd, .d_release = ll_release, .d_delete = ll_ddelete, + .d_compare = ll_dcompare, };
Signature · 1 line
1.8.3.1
PATCH 30/45 lustre: fallocate: Implement fallocate preallocate operation
In reply to James SimmonsFrom: Swapnil Pimpale <[email protected]>
This patch adds fallocate(2) preallocate operation support for Lustre. fallocate(2) method of the inode_operations or file_operations is implemented and transported to the OSTs to interface with the underlying OSD's fallocate(2) code.
In a saperate patch, a new RPC, OST_FALLOCATE has been added and reserved for space preallocation. The fallocate functionality (prealloc) in CLIO has been multiplexed with CIT_SETATTR. (https://review.whamcloud.com/37277)
Lustre fsx(File system exerciser) is updated in a saperate patch to handle fallocate calls. (https://review.whamcloud.com/37277)
Only fallocate preallocate operation is supported by this patch for now. Other operations like, FALLOC_FL_PUNCH (deallocate), FALLOC_FL_ZERO_RANGE, FALLOC_FL_COLLAPSE_RANGE and FALLOC_FL_INSPECT_RANGE is not supported by this patch and will be addressed by a separate patch.
ZFS operation is not supported by this patch. ZFS fallocate(2) will be addressed by patch (https://review.whamcloud.com/36506/)
New test case under sanity is added to verify fallocate call.
WC-bug-id: https://jira.whamcloud.com/browse/LU-3606
Lustre-commit: 48457868a02ae ("LU-3606 fallocate: Implement fallocate preallocate operation")
Reviewed-on: https://review.whamcloud.com/9275
--- fs/lustre/include/cl_object.h | 44 +++++++++++++++++- fs/lustre/include/lustre_req_layout.h | 1 + fs/lustre/include/obd_support.h | 5 ++- fs/lustre/llite/file.c | 84 +++++++++++++++++++++++++++++++++-- fs/lustre/llite/lcommon_cl.c | 2 + fs/lustre/llite/llite_internal.h | 1 + fs/lustre/llite/lproc_llite.c | 1 + fs/lustre/llite/vvp_io.c | 17 ++++--- fs/lustre/lov/lov_io.c | 27 ++++++++--- fs/lustre/osc/osc_internal.h | 3 ++ fs/lustre/osc/osc_io.c | 56 +++++++++++++++++++---- fs/lustre/osc/osc_request.c | 65 ++++++++++++++++++++++++++- fs/lustre/ptlrpc/layout.c | 5 +++ 13 files changed, 286 insertions(+), 25 deletions(-)
fs/lustre/include/cl_object.h +43 -1
@@ -89,6 +89,7 @@/* * super-class definitions. */ +#include <linux/uio.h> #include <lu_object.h> #include <linux/atomic.h> #include <linux/mutex.h>
@@ -1765,6 +1766,32 @@ struct cl_io_rw_common {int crw_nonblock; }; +enum cl_setattr_subtype { +/** regular setattr **/ +CL_SETATTR_REG =1 , +/** truncate(2) **/ +CL_SETATTR_TRUNC , +/** fallocate(2) - mode preallocate **/ +CL_SETATTR_FALLOCATE +}; + +struct cl_io_range { +loff_t cir_pos; +size_t cir_count; +}; + +struct cl_io_pt { +struct cl_io_pt *cip_next; +struct kiocb cip_iocb; +struct iov_iter cip_iter; +struct file *cip_file; +enum cl_io_type cip_iot; +unsigned int cip_need_restart:1 ; +loff_t cip_pos; +size_t cip_count; +ssize_t cip_result; +}; +/** * State for io. *
@@ -1812,6 +1839,14 @@ struct cl_io {int sa_stripe_index;struct ost_layout sa_layout;const struct lu_fid *sa_parent_fid; +/* SETATTR interface is used for regular setattr, */ +/* truncate(2) and fallocate(2) subtypes */ +enum cl_setattr_subtype sa_subtype; +/* The following are used for fallocate(2) */ +int sa_falloc_mode; +loff_t sa_falloc_offset; +loff_t sa_falloc_len; +loff_t sa_falloc_end; } ci_setattr;struct cl_data_version_io {u64 dv_data_version;
@@ -2399,7 +2434,14 @@ static inline int cl_io_is_mkwrite(const struct cl_io *io)static inline int cl_io_is_trunc (const struct cl_io *io) {return io->ci_type ==CIT_SETATTR && - (io->u.ci_setattr.sa_avalid &ATTR_SIZE ); + (io->u.ci_setattr.sa_avalid &ATTR_SIZE ) && + (io->u.ci_setattr.sa_subtype !=CL_SETATTR_FALLOCATE ); +} + +static inline int cl_io_is_fallocate (const struct cl_io *io) +{ +return (io->ci_type ==CIT_SETATTR ) && + (io->u.ci_setattr.sa_subtype ==CL_SETATTR_FALLOCATE ); }struct cl_io *cl_io_top (struct cl_io *io);
fs/lustre/include/lustre_req_layout.h +1 -0
@@ -186,6 +186,7 @@ void req_capsule_shrink(struct req_capsule *pill,extern struct req_format RQF_OST_SETATTR ;extern struct req_format RQF_OST_CREATE ;extern struct req_format RQF_OST_PUNCH ; +extern struct req_format RQF_OST_FALLOCATE ;extern struct req_format RQF_OST_SYNC ;extern struct req_format RQF_OST_DESTROY ;extern struct req_format RQF_OST_BRW_READ ;
fs/lustre/include/obd_support.h +3 -2
@@ -264,8 +264,9 @@#define OBD_FAIL_OST_DQACQ_NET 0x230#define OBD_FAIL_OST_STATFS_EINPROGRESS 0x231#define OBD_FAIL_OST_SET_INFO_NET 0x232 -#define OBD_FAIL_OST_DISCONNECT_DELAY 0x245 -#define OBD_FAIL_OST_PREPARE_DELAY 0x247 +#define OBD_FAIL_OST_DISCONNECT_DELAY 0x245 +#define OBD_FAIL_OST_PREPARE_DELAY 0x247 +#define OBD_FAIL_OST_FALLOCATE_NET 0x249#define OBD_FAIL_LDLM 0x300#define OBD_FAIL_LDLM_NAMESPACE_NEW 0x301
fs/lustre/llite/file.c +81 -3
114 lines not on this page · raw message
fs/lustre/llite/lcommon_cl.c +2 -0
@@ -102,6 +102,8 @@ int cl_setattr_ost(struct cl_object *obj, const struct iattr *attr, io->u.ci_setattr.sa_xvalid = xvalid; io->u.ci_setattr.sa_parent_fid =lu_object_fid (&obj->co_lu); +if (attr->ia_valid &ATTR_SIZE ) + io->u.ci_setattr.sa_subtype =CL_SETATTR_TRUNC ; again:if (attr->ia_valid &ATTR_FILE )ll_io_set_mirror (io, attr->ia_file);
fs/lustre/llite/llite_internal.h +1 -0
@@ -916,6 +916,7 @@ enum {LPROC_LL_LISTXATTR ,LPROC_LL_REMOVEXATTR ,LPROC_LL_INODE_PERM , +LPROC_LL_FALLOCATE , LPROC_LL_FILE_OPCODES };
fs/lustre/llite/lproc_llite.c +1 -0
@@ -1580,6 +1580,7 @@ static void sbi_kobj_release(struct kobject *kobj) { LPROC_LL_TRUNC, LPROCFS_TYPE_LATENCY, "truncate" }, { LPROC_LL_FLOCK, LPROCFS_TYPE_LATENCY, "flock" }, { LPROC_LL_GETATTR, LPROCFS_TYPE_LATENCY, "getattr" }, + { LPROC_LL_FALLOCATE, LPROCFS_TYPE_LATENCY, "fallocate" },/* dir inode operation */ { LPROC_LL_CREATE, LPROCFS_TYPE_LATENCY, "create" }, { LPROC_LL_LINK, LPROCFS_TYPE_LATENCY, "link" },
fs/lustre/llite/vvp_io.c +12 -5
@@ -615,13 +615,16 @@ static int vvp_io_setattr_lock(const struct lu_env *env, const struct cl_io_slice *ios) {struct cl_io *io = ios->cis_io; -u64 new_size; +u64 lock_start =0 ; +u64 lock_end =OBD_OBJECT_EOF ;u32 enqflags =0 ;if (cl_io_is_trunc (io)) { - new_size = io->u.ci_setattr.sa_attr.lvb_size; -if (new_size ==0 ) +if (io->u.ci_setattr.sa_attr.lvb_size ==0 ) enqflags =CEF_DISCARD_DATA ; + }else if (cl_io_is_fallocate (io)) { + lock_start = io->u.ci_setattr.sa_falloc_offset; + lock_end = lock_start + io->u.ci_setattr.sa_attr.lvb_size; }else {unsigned int valid = io->u.ci_setattr.sa_avalid;
@@ -635,11 +638,10 @@ static int vvp_io_setattr_lock(const struct lu_env *env, io->u.ci_setattr.sa_attr.lvb_atime >= io->u.ci_setattr.sa_attr.lvb_ctime))return 0 ; - new_size =0 ; }return vvp_io_one_lock (env, io, enqflags,CLM_WRITE , - new_size,OBD_OBJECT_EOF ); + lock_start, lock_end); }static int vvp_do_vmtruncate (struct inode *inode,size_t size)
@@ -695,6 +697,9 @@ static int vvp_io_setattr_start(const struct lu_env *env,trunc_sem_down_write (&lli->lli_trunc_sem);inode_lock (inode);inode_dio_wait (inode); + }else if (cl_io_is_fallocate (io)) { +inode_lock (inode); +inode_dio_wait (inode); }else {inode_lock (inode); }
@@ -719,6 +724,8 @@ static void vvp_io_setattr_end(const struct lu_env *env,vvp_do_vmtruncate (inode, io->u.ci_setattr.sa_attr.lvb_size);inode_unlock (inode);trunc_sem_up_write (&lli->lli_trunc_sem); + }else if (cl_io_is_fallocate (io)) { +inode_unlock (inode); }else {inode_unlock (inode); }
fs/lustre/lov/lov_io.c +22 -5
@@ -486,11 +486,16 @@ static int lov_io_slice_init(struct lov_io *lio, struct lov_object *obj,break ;case CIT_SETATTR : -if (cl_io_is_trunc (io)) +if (cl_io_is_fallocate (io)) { + lio->lis_pos = io->u.ci_setattr.sa_falloc_offset; + lio->lis_endpos = io->u.ci_setattr.sa_falloc_end; + }else if (cl_io_is_trunc (io)) { lio->lis_pos = io->u.ci_setattr.sa_attr.lvb_size; -else + lio->lis_endpos =OBD_OBJECT_EOF ; + }else { lio->lis_pos =0 ; - lio->lis_endpos =OBD_OBJECT_EOF ; + lio->lis_endpos =OBD_OBJECT_EOF ; + }break ;case CIT_DATA_VERSION :
@@ -639,15 +644,24 @@ static void lov_io_sub_inherit(struct lov_io_sub *sub, struct lov_io *lio, parent->u.ci_setattr.sa_attr_flags; io->u.ci_setattr.sa_avalid = parent->u.ci_setattr.sa_avalid; io->u.ci_setattr.sa_xvalid = parent->u.ci_setattr.sa_xvalid; + io->u.ci_setattr.sa_falloc_mode = + parent->u.ci_setattr.sa_falloc_mode; io->u.ci_setattr.sa_stripe_index = stripe; io->u.ci_setattr.sa_parent_fid = parent->u.ci_setattr.sa_parent_fid; +/* For SETATTR(fallocate) pass the subtype to lower IO */ + io->u.ci_setattr.sa_subtype = parent->u.ci_setattr.sa_subtype;if (cl_io_is_trunc (io)) {loff_t new_size = parent->u.ci_setattr.sa_attr.lvb_size; new_size =lov_size_to_stripe (lsm, index, new_size, stripe); io->u.ci_setattr.sa_attr.lvb_size = new_size; + }else if (cl_io_is_fallocate (io)) { + io->u.ci_setattr.sa_falloc_offset = start; + io->u.ci_setattr.sa_falloc_end = end; + io->u.ci_setattr.sa_attr.lvb_size = + parent->u.ci_setattr.sa_attr.lvb_size; }lov_lsm2layout (lsm, lsm->lsm_entries[index], &io->u.ci_setattr.sa_layout);
@@ -1488,8 +1502,11 @@ int lov_io_init_released(const struct lu_env *env, struct cl_object *obj,* - in open, for open O_TRUNC * - in setattr, for truncate */ -/* the truncate is for size > 0 so triggers a restore */ -if (cl_io_is_trunc (io)) { +/* +* the truncate is for size > 0 so triggers a restore, +* also trigger a restore for prealloc/punch +*/ +if (cl_io_is_trunc (io) ||cl_io_is_fallocate (io)) { io->ci_restore_needed =1 ; result = -ENODATA ; }else {
fs/lustre/osc/osc_internal.h +3 -0
@@ -73,6 +73,9 @@ int osc_match_base(const struct lu_env *env, struct obd_export *exp,int osc_setattr_async (struct obd_export *exp,struct obdo *oa,obd_enqueue_update_f upcall,void *cookie,struct ptlrpc_request_set *rqset); +int osc_fallocate_base (struct obd_export *exp,struct obdo *oa, +obd_enqueue_update_f upcall,void *cookie, +int mode);int osc_sync_base (struct osc_object *exp,struct obdo *oa,obd_enqueue_update_f upcall,void *cookie,struct ptlrpc_request_set *rqset);
fs/lustre/osc/osc_io.c +48 -8
104 lines not on this page · raw message
fs/lustre/osc/osc_request.c +64 -1
78 lines not on this page · raw message
fs/lustre/ptlrpc/layout.c +5 -0
@@ -757,6 +757,7 @@ &RQF_OST_SETATTR , &RQF_OST_CREATE , &RQF_OST_PUNCH , + &RQF_OST_FALLOCATE , &RQF_OST_SYNC , &RQF_OST_DESTROY , &RQF_OST_BRW_READ,
@@ -1595,6 +1596,10 @@ struct req_format RQF_OST_PUNCH =DEFINE_REQ_FMT0 ("OST_PUNCH" , ost_body_capa, ost_body_only);EXPORT_SYMBOL (RQF_OST_PUNCH ); +struct req_format RQF_OST_FALLOCATE = +DEFINE_REQ_FMT0 ("OST_FALLOCATE" , ost_body_capa, ost_body_only); +EXPORT_SYMBOL (RQF_OST_FALLOCATE ); +struct req_format RQF_OST_SYNC =DEFINE_REQ_FMT0 ("OST_SYNC" , ost_body_capa, ost_body_only);EXPORT_SYMBOL (RQF_OST_SYNC );
Signature · 1 line
1.8.3.1
PATCH 31/45 lustre: llite: fix possible divide zero in ll_use_fast_io()
In reply to James SimmonsFrom: Wang Shilong <[email protected]>
ll_use_fast_io() is used to check wheather we could use fast IO. Since it is called in fast path, we don't hold ras_lock to protect access, there might have the race @ras_stride_bytes is reset after stride_io_mode() check.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13541
Lustre-commit: 7cd0afe58321 ("LU-13541 llite: fix possible divide zero in ll_use_fast_io()")
Reviewed-on: https://review.whamcloud.com/38545
--- fs/lustre/llite/rw.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
fs/lustre/llite/rw.c +3 -2
@@ -1609,10 +1609,11 @@ static bool ll_use_fast_io(struct file *file,unsigned long fast_read_pages =max (RA_REMAIN_WINDOW_MIN , ras->ras_rpc_pages);loff_t skip_pages; +loff_t stride_bytes = ras->ras_stride_bytes; -if (stride_io_mode (ras)) { +if (stride_io_mode (ras) && stride_bytes) { skip_pages = (ras->ras_stride_length + - ras->ras_stride_bytes -1 ) / ras->ras_stride_bytes; + ras->ras_stride_bytes -1 ) / stride_bytes; skip_pages *= fast_read_pages; }else { skip_pages = fast_read_pages;
Signature · 1 line
1.8.3.1
PATCH 32/45 lustre: llog: allow delete of zero size llog
In reply to James SimmonsFrom: Alexander Boyko <[email protected]>
1) all plain logs belonging to catalog should have flag LLOG_F_ZAP_WHEN_EMPTY base on llog_cat_new_log(). When llog_cat_process_common processing plain log with zero file size, this flag is not set during llog_cat_id2handle LLOG_EMPTY, so these plain llogs are not canceled/destroyed. They appeared during cross MDT updates. Fix adds flag LLOG_F_ZAP_WHEN_EMPTY for any plain llog at catalog.
Cray-bug-id: LUS-8634
WC-bug-id: https://jira.whamcloud.com/browse/LU-13411
Lustre-commit: bc7f8cfe0fc6a ("LU-13411 llog: allow delete of zero size llog")
Reviewed-on: https://review.whamcloud.com/38131
--- fs/lustre/obdclass/llog.c | 9 +++++---- fs/lustre/obdclass/llog_cat.c | 3 ++- 2 files changed, 7 insertions(+), 5 deletions(-)
fs/lustre/obdclass/llog.c +5 -4
@@ -472,6 +472,7 @@ int llog_process_or_fork(const struct lu_env *env,struct llog_process_info *lpi;struct llog_process_data *d = data;struct llog_process_cat_data *cd = catdata; +u32 flags = loghandle->lgh_hdr->llh_flags;int rc; lpi =kzalloc (sizeof (*lpi),GFP_KERNEL );
@@ -484,10 +485,10 @@ int llog_process_or_fork(const struct lu_env *env,CDEBUG (D_OTHER ,"Processing " DFID " flags 0x%03x startcat %d startidx %d first_idx %d last_idx %d\n" , -PFID (&loghandle->lgh_id.lgl_oi.oi_fid), - loghandle->lgh_hdr->llh_flags, d ? d->lpd_startcat :-1 , - d ? d->lpd_startidx :-1 , cd ? cd->lpcd_first_idx :-1 , - cd ? cd->lpcd_last_idx :-1 ); +PFID (&loghandle->lgh_id.lgl_oi.oi_fid), flags, + (flags &LLOG_F_IS_CAT ) && d ? d->lpd_startcat :-1 , + (flags &LLOG_F_IS_CAT ) && d ? d->lpd_startidx :-1 , + cd ? cd->lpcd_first_idx :-1 , cd ? cd->lpcd_last_idx :-1 );if (fork) {struct task_struct *task;
fs/lustre/obdclass/llog_cat.c +2 -1
@@ -108,7 +108,8 @@ static int llog_cat_id2handle(const struct lu_env *env,return rc; } - rc =llog_init_handle (env, loghandle, fmt |LLOG_F_IS_PLAIN ,NULL ); + rc =llog_init_handle (env, loghandle,LLOG_F_IS_PLAIN | +LLOG_F_ZAP_WHEN_EMPTY | fmt,NULL );if (rc <0 ) {llog_close (env, loghandle); *res =NULL ;
Signature · 1 line
1.8.3.1
PATCH 33/45 lustre: ldlm: use proper units for timeouts
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
Use ktime_t for ns_dirty_age_limit internally, even though the user interface is in seconds, since this is frequenty used together with other ktime_t values in the kernel.
WC-bug-id: https://jira.whamcloud.com/browse/LU-12931
Lustre-commit: 3108bbb0b8485 ("LU-12931 ldlm: use proper units for timeouts")
Reviewed-on: https://review.whamcloud.com/38365
--- fs/lustre/include/lustre_dlm.h | 15 +++++++++------ fs/lustre/ldlm/ldlm_lockd.c | 3 +-- fs/lustre/ldlm/ldlm_resource.c | 7 ++++--- fs/lustre/llite/namei.c | 3 +-- 4 files changed, 15 insertions(+), 13 deletions(-)
fs/lustre/include/lustre_dlm.h +9 -6
@@ -415,15 +415,18 @@ struct ldlm_namespace {*/ unsigned int ns_max_unused; -/** Maximum allowed age (last used time) for locks in the LRU */ +/** Maximum allowed age (last used time) for locks in the LRU. Set in +* seconds from userspace, but stored in ns to avoid repeat conversions. +*/ ktime_t ns_max_age;/** -* Number of seconds since the lock was last used. The client may -* cancel the lock limited by this age and flush related data if -* any other client shows interest in it doing glimpse request. -* This allows to cache stat data locally for such files early. +* Number of (nano)seconds since the lock was last used. The client +* may cancel the lock older than this age and flush related data if +* another client shows interest in this lock by doing glimpse request. +* This allows to cache stat data locally for such files early. Set in +* seconds from userspace, but stored in ns to avoid repeat conversions. */ -time64_t ns_dirty_age_limit; +ktime_t ns_dirty_age_limit;/** * Used to rate-limit ldlm_namespace_dump calls. * \see ldlm_namespace_dump. Increased by 10 seconds every time
fs/lustre/ldlm/ldlm_lockd.c +1 -2
@@ -378,8 +378,7 @@ static void ldlm_handle_gl_callback(struct ptlrpc_request *req,if (lock->l_granted_mode ==LCK_PW && !lock->l_readers && !lock->l_writers &&ktime_after (ktime_get (), -ktime_add (lock->l_last_used, -ktime_set (ns->ns_dirty_age_limit,0 )))) { +ktime_add (lock->l_last_used, ns->ns_dirty_age_limit))) {unlock_res_and_lock (lock);/* For MDS glimpse it is always DOM lock, set corresponding
fs/lustre/ldlm/ldlm_resource.c +4 -3
@@ -332,7 +332,8 @@ static ssize_t dirty_age_limit_show(struct kobject *kobj, struct ldlm_namespace *ns = container_of(kobj, struct ldlm_namespace, ns_kobj); -return sprintf (buf,"%llu\n" , ns->ns_dirty_age_limit); +return scnprintf (buf,PAGE_SIZE ,"%llu\n" , +ktime_divns (ns->ns_dirty_age_limit,NSEC_PER_SEC )); } static ssize_t dirty_age_limit_store(struct kobject *kobj,
@@ -346,7 +347,7 @@ static ssize_t dirty_age_limit_store(struct kobject *kobj,if (kstrtoull (buffer,10 , &tmp))return -EINVAL ; - ns->ns_dirty_age_limit = tmp; + ns->ns_dirty_age_limit =ktime_set (tmp,0 );return count; }
@@ -646,7 +647,7 @@ struct ldlm_namespace *ldlm_namespace_new(struct obd_device *obd, char *name, ns->ns_max_age =ktime_set (LDLM_DEFAULT_MAX_ALIVE ,0 ); ns->ns_orig_connect_flags =0 ; ns->ns_connect_flags =0 ; - ns->ns_dirty_age_limit =LDLM_DIRTY_AGE_LIMIT ; + ns->ns_dirty_age_limit =ktime_set (LDLM_DIRTY_AGE_LIMIT ,0 ); ns->ns_stopping =0 ; ns->ns_last_pos = &ns->ns_unused_list;
fs/lustre/llite/namei.c +1 -2
@@ -418,8 +418,7 @@ int ll_md_need_convert(struct ldlm_lock *lock)/* is lock is too old to be converted? */ lock_res_and_lock (lock);if (ktime_after (ktime_get (), -ktime_add (lock->l_last_used, -ktime_set (ns->ns_dirty_age_limit,0 )))) { +ktime_add (lock->l_last_used, ns->ns_dirty_age_limit))) {unlock_res_and_lock (lock);return 0 ; }
Signature · 1 line
1.8.3.1
PATCH 34/45 lustre: dne: support directory restripe
In reply to James SimmonsFrom: Lai Siyao <[email protected]>
This patch adds directory restripe support:
* 'lfs setdirstripe -m -1 -c <stripe_count>' on an existed directory will change this directory layout, if 'stripe_count' is larger than current count, new stripes are allocated after current stripes, otherwise merge stripes of this directory, NB, if stripe count is unchanged, but hash type changed, it's treated as merging, but rehashing actually.
* mdt_restripe() ia added to restripe directory.
* mdd_dir_declare_layout_split() is added to split directory, which handles both plain and striped directory split.
* lod_dir_declare_layout_split() will handle the internal of directory split.
* directory merge is simple compared to split, which just records target stripe count in LMV, and update it.
NB. this patch only restripe directory, but doesn't add the code to migrate sub files, which will be implemented in the following patch.
WC-bug-id: https://jira.whamcloud.com/browse/LU-11025
Lustre-commit: 2e2b16c28bcf4 ("LU-11025 dne: support directory restripe")
Reviewed-on: https://review.whamcloud.com/36898
--- fs/lustre/include/lustre_lmv.h | 90 ++++++++++++++++++++++++++++++--- fs/lustre/lmv/lmv_obd.c | 57 ++++++++++++++------- fs/lustre/ptlrpc/wiretest.c | 6 +++ include/uapi/linux/lustre/lustre_user.h | 37 +++++++++++++- 4 files changed, 166 insertions(+), 24 deletions(-)
fs/lustre/include/lustre_lmv.h +84 -6
@@ -68,7 +68,7 @@ static inline bool lmv_dir_foreign(const struct lmv_stripe_md *lsm)static inline bool lmv_dir_layout_changing (const struct lmv_stripe_md *lsm) {return lmv_dir_striped (lsm) && - (lsm->lsm_md_hash_type &LMV_HASH_FLAG_LAYOUT_CHANGE ); +lmv_hash_is_layout_changing (lsm->lsm_md_hash_type); }static inline bool lmv_dir_bad_hash (const struct lmv_stripe_md *lsm)
@@ -275,6 +275,15 @@ static inline u32 crush_hash(u32 a, u32 b)return idx; } +/* directory layout may change in three ways: +* 1. directory migration, in its LMV source stripes are appended after +* target stripes, @migrate_hash is source hash type, @migrate_offset is +* target stripe count, +* 2. directory split, @migrate_hash is hash type before split, +* @migrate_offset is stripe count before split. +* 3. directory merge, @migrate_hash is hash type after merge, +* @migrate_offset is stripe count after merge. +*/ static inline int __lmv_name_to_stripe_index(u32 hash_type, u32 stripe_count, u32 migrate_hash, u32 migrate_offset,
@@ -287,7 +296,17 @@ static inline u32 crush_hash(u32 a, u32 b)LASSERT (namelen >0 );LASSERT (stripe_count >0 ); -if (hash_type &LMV_HASH_FLAG_MIGRATION ) { +if (lmv_hash_is_splitting (hash_type)) { +if (!new_layout) { + hash_type = migrate_hash; + stripe_count = migrate_offset; + } + }else if (lmv_hash_is_merging (hash_type)) { +if (new_layout) { + hash_type = migrate_hash; + stripe_count = migrate_offset; + } + }else if (lmv_hash_is_migrating (hash_type)) {if (new_layout) { stripe_count = migrate_offset; }else {
@@ -317,12 +336,12 @@ static inline u32 crush_hash(u32 a, u32 b)LASSERT (stripe_index < stripe_count); -if ((saved_hash &LMV_HASH_FLAG_MIGRATION ) && !new_layout) +if (!new_layout &&lmv_hash_is_migrating (saved_hash)) stripe_index += migrate_offset;LASSERT (stripe_index < saved_count); -CDEBUG (D_INFO ,"name %.*s hash %#x/%#x idx %d/%u/%u under %s layout\n" , +CDEBUG (D_INFO ,"name %.*s hash=%#x/%#x idx=%d/%u/%u under %s layout\n" , namelen, name, saved_hash, migrate_hash, stripe_index, saved_count, migrate_offset, new_layout ?"new" :"old" );
@@ -382,21 +401,25 @@ static inline bool lmv_user_magic_supported(u32 lum_magic)#define LMV_DEBUG (mask, lmv, msg) \ CDEBUG(mask, \ - "%s LMV: magic %#x count %u index %u hash %#x version %u migrate offset %u migrate hash %u.\n",\ + "%s LMV: magic=%#x count=%u index=%u hash=%#x version=%u migrate offset=%u migrate hash=%u.\n",\ msg, (lmv)->lmv_magic, (lmv)->lmv_stripe_count, \ (lmv)->lmv_master_mdt_index, (lmv)->lmv_hash_type, \ (lmv)->lmv_layout_version, (lmv)->lmv_migrate_offset, \ (lmv)->lmv_migrate_hash) +/* master LMV is sane */ static inline bool lmv_is_sane (const struct lmv_mds_md_v1 *lmv) { +if (!lmv) +return false; +if (le32_to_cpu (lmv->lmv_magic) !=LMV_MAGIC_V1 ) goto insane;if (le32_to_cpu (lmv->lmv_stripe_count) ==0 ) goto insane; -if (!lmv_is_known_hash_type (lmv->lmv_hash_type)) +if (!lmv_is_known_hash_type (le32_to_cpu (lmv->lmv_hash_type))) goto insane;return true;
@@ -405,4 +428,59 @@ static inline bool lmv_is_sane(const struct lmv_mds_md_v1 *lmv) return false; } +/* LMV can be either master or stripe LMV */ +static inline bool lmv_is_sane2 (const struct lmv_mds_md_v1 *lmv) +{ +if (!lmv) +return false; + +if (le32_to_cpu (lmv->lmv_magic) !=LMV_MAGIC_V1 && +le32_to_cpu (lmv->lmv_magic) !=LMV_MAGIC_STRIPE ) + goto insane; + +if (le32_to_cpu (lmv->lmv_stripe_count) ==0 ) + goto insane; + +if (!lmv_is_known_hash_type (le32_to_cpu (lmv->lmv_hash_type))) + goto insane; + +return true; +insane: +LMV_DEBUG (D_ERROR , lmv,"insane" ); +return false; +} + +static inline bool lmv_is_splitting (const struct lmv_mds_md_v1 *lmv) +{ +LASSERT (lmv_is_sane2 (lmv)); +return lmv_hash_is_splitting (cpu_to_le32 (lmv->lmv_hash_type)); +} + +static inline bool lmv_is_merging (const struct lmv_mds_md_v1 *lmv) +{ +LASSERT (lmv_is_sane2 (lmv)); +return lmv_hash_is_merging (cpu_to_le32 (lmv->lmv_hash_type)); +} + +static inline bool lmv_is_migrating (const struct lmv_mds_md_v1 *lmv) +{ +LASSERT (lmv_is_sane (lmv)); +return lmv_hash_is_migrating (cpu_to_le32 (lmv->lmv_hash_type)); +} + +static inline bool lmv_is_restriping (const struct lmv_mds_md_v1 *lmv) +{ +LASSERT (lmv_is_sane2 (lmv)); +return lmv_hash_is_splitting (cpu_to_le32 (lmv->lmv_hash_type)) || +lmv_hash_is_merging (cpu_to_le32 (lmv->lmv_hash_type)); +} + +static inline bool lmv_is_layout_changing (const struct lmv_mds_md_v1 *lmv) +{ +LASSERT (lmv_is_sane2 (lmv)); +return lmv_hash_is_splitting (cpu_to_le32 (lmv->lmv_hash_type)) || +lmv_hash_is_merging (cpu_to_le32 (lmv->lmv_hash_type)) || +lmv_hash_is_migrating (cpu_to_le32 (lmv->lmv_hash_type)); +} +#endif
fs/lustre/lmv/lmv_obd.c +40 -17
83 lines not on this page · raw message
fs/lustre/ptlrpc/wiretest.c +6 -0
@@ -1760,6 +1760,10 @@ void lustre_assert_wire_constants(void)BUILD_BUG_ON (LMV_MAGIC_V1 !=0x0CD20CD0 );BUILD_BUG_ON (LMV_MAGIC_STRIPE !=0x0CD40CD0 );BUILD_BUG_ON (LMV_HASH_TYPE_MASK !=0x0000ffff ); +BUILD_BUG_ON (LMV_HASH_FLAG_MERGE !=0x04000000 ); +BUILD_BUG_ON (LMV_HASH_FLAG_SPLIT !=0x08000000 ); +BUILD_BUG_ON (LMV_HASH_FLAG_LOST_LMV !=0x10000000 ); +BUILD_BUG_ON (LMV_HASH_FLAG_BAD_TYPE !=0x20000000 );BUILD_BUG_ON (LMV_HASH_FLAG_MIGRATION !=0x80000000 );BUILD_BUG_ON (LMV_CRUSH_PG_COUNT !=4096 );
@@ -2075,6 +2079,8 @@ void lustre_assert_wire_constants(void) (unsigned int)MDS_OWNEROVERRIDE);LASSERTF (MDS_HSM_RELEASE ==0x00001000UL ,"found 0x%.8xUL\n" , (unsigned int )MDS_HSM_RELEASE ); +LASSERTF (MDS_CLOSE_MIGRATE ==0x00002000UL ,"found 0x%.8xUL\n" , + (unsigned int )MDS_CLOSE_MIGRATE );LASSERTF (MDS_CLOSE_LAYOUT_SWAP ==0x00004000UL ,"found 0x%.8xUL\n" , (unsigned int )MDS_CLOSE_LAYOUT_SWAP ); LASSERTF(MDS_CLOSE_LAYOUT_MERGE == 0x00008000UL, "found 0x%.8xUL\n",
include/uapi/linux/lustre/lustre_user.h +36 -1
@@ -706,6 +706,9 @@ static inline bool lmv_is_known_hash_type(__u32 type) (type &LMV_HASH_TYPE_MASK ) ==LMV_HASH_TYPE_CRUSH ; } +#define LMV_HASH_FLAG_MERGE 0x04000000 +#define LMV_HASH_FLAG_SPLIT 0x08000000 +/* The striped directory has ever lost its master LMV EA, then LFSCK * re-generated it. This flag is used to indicate such case. It is an * on-disk flag.
@@ -715,7 +718,39 @@ static inline bool lmv_is_known_hash_type(__u32 type)#define LMV_HASH_FLAG_BAD_TYPE 0x20000000#define LMV_HASH_FLAG_MIGRATION 0x80000000 -#define LMV_HASH_FLAG_LAYOUT_CHANGE LMV_HASH_FLAG_MIGRATION +#define LMV_HASH_FLAG_LAYOUT_CHANGE \ + (LMV_HASH_FLAG_MIGRATION | LMV_HASH_FLAG_SPLIT | LMV_HASH_FLAG_MERGE) + +/* both SPLIT and MIGRATION are set for directory split */ +static inline bool lmv_hash_is_splitting (__u32 hash) +{ +return (hash &LMV_HASH_FLAG_LAYOUT_CHANGE ) == + (LMV_HASH_FLAG_SPLIT |LMV_HASH_FLAG_MIGRATION ); +} + +/* both MERGE and MIGRATION are set for directory merge */ +static inline bool lmv_hash_is_merging (__u32 hash) +{ +return (hash &LMV_HASH_FLAG_LAYOUT_CHANGE ) == + (LMV_HASH_FLAG_MERGE |LMV_HASH_FLAG_MIGRATION ); +} + +/* only MIGRATION is set for directory migration */ +static inline bool lmv_hash_is_migrating (__u32 hash) +{ +return (hash &LMV_HASH_FLAG_LAYOUT_CHANGE ) ==LMV_HASH_FLAG_MIGRATION ; +} + +static inline bool lmv_hash_is_restriping (__u32 hash) +{ +return lmv_hash_is_splitting (hash) ||lmv_hash_is_merging (hash); +} + +static inline bool lmv_hash_is_layout_changing (__u32 hash) +{ +return lmv_hash_is_splitting (hash) ||lmv_hash_is_merging (hash) || +lmv_hash_is_migrating (hash); +}struct lustre_foreign_type {__u32 lft_type;
Signature · 1 line
1.8.3.1
PATCH 35/45 lustre: osc: Do not wait for grants for too long
In reply to James SimmonsFrom: Oleg Drokin <[email protected]>
obd_timeout is way too long considering we are holding a lock that might be contended. If OST is slow to respond, we might get evicted, so limit us to a half of the shortest possible max wait a server might have before switching to synchronous IO.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13131
Lustre-commit: 1eee11c75ca13 ("LU-13131 osc: Do not wait for grants for too long")
Reviewed-on: https://review.whamcloud.com/38283
--- fs/lustre/include/lustre_dlm.h | 2 ++ fs/lustre/ldlm/ldlm_request.c | 1 + fs/lustre/osc/osc_cache.c | 13 ++++++++++++- 3 files changed, 15 insertions(+), 1 deletion(-)
fs/lustre/include/lustre_dlm.h +2 -0
@@ -1320,6 +1320,8 @@ int ldlm_cli_cancel_list(struct list_head *head, int count, enum ldlm_cancel_flags flags);/** @} ldlm_cli_api */ +extern unsigned int ldlm_enqueue_min; +int ldlm_inodebits_drop (struct ldlm_lock *lock,u64 to_drop);int ldlm_cli_inodebits_convert (struct ldlm_lock *lock,enum ldlm_cancel_flags cancel_flags);
fs/lustre/ldlm/ldlm_request.c +1 -0
@@ -69,6 +69,7 @@unsigned int ldlm_enqueue_min =OBD_TIMEOUT_DEFAULT ;module_param (ldlm_enqueue_min, uint,0644 );MODULE_PARM_DESC (ldlm_enqueue_min,"lock enqueue timeout minimum" ); +EXPORT_SYMBOL (ldlm_enqueue_min);/* in client side, whether the cached locks will be canceled before replay */ unsigned int ldlm_cancel_unused_locks_before_replay =1 ;
fs/lustre/osc/osc_cache.c +12 -1
@@ -39,6 +39,7 @@#define DEBUG_SUBSYSTEM S_OSC#include <lustre_osc.h> +#include <lustre_dlm.h> #include "osc_internal.h"
@@ -1630,10 +1631,20 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, {struct osc_object *osc = oap->oap_obj;struct lov_oinfo *loi = osc->oo_oinfo; -unsigned long timeout = (AT_OFF ? obd_timeout : at_max) *HZ ;int rc = -EDQUOT ;int remain;bool entered = false; +/* We cannot wait for a long time here since we are holding ldlm lock +* across the actual IO. If no requests complete fast (e.g. due to +* overloaded OST that takes a long time to process everything, we'd +* get evicted if we wait for a normal obd_timeout or some such. +* So we try to wait half the time it would take the client to be +* evicted by server which is half obd_timeout when AT is off +* or at least ldlm_enqueue_min with AT on. +* See LU-13131 +*/ +unsigned long timeout = (AT_OFF ? obd_timeout /2 : + ldlm_enqueue_min /2 ) *HZ ;OSC_DUMP_GRANT (D_CACHE , cli,"need:%d\n" , bytes);
Signature · 1 line
1.8.3.1
PATCH 36/45 lnet: use kmem_cache_zalloc as appropriate.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
Rather than passing __GFP_ZERO to kmem_cache_alloc(), or calling memset(0) after the allocation, use kmem_cache_zalloc().
kmem_cache_zalloc() has been part of Linux since 2.6.17.
WC-bug-id: https://jira.whamcloud.com/browse/LU-6142
Lustre-commit: 3ceed00b0458d ("LU-6142 kernel: use kmem_cache_zalloc as appropriate.")
Reviewed-on: https://review.whamcloud.com/38439
--- net/lnet/lnet/lib-me.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/lnet/lnet/lib-me.c +1 -1
@@ -87,7 +87,7 @@ struct lnet_me *if (!mtable)/* can't match portal type */ return ERR_PTR (-EPERM ); - me =kmem_cache_alloc (lnet_mes_cachep,GFP_NOFS | __GFP_ZERO); + me =kmem_cache_zalloc (lnet_mes_cachep,GFP_NOFS );if (!me)return ERR_PTR (-ENOMEM );
Signature · 1 line
1.8.3.1
PATCH 37/45 lustre: osc: Ensure immediate departure of sync write pages
In reply to James SimmonsFrom: Oleg Drokin <[email protected]>
Except for the case of direct-io and server-lock, we are hold potentially multiple locks that are next to impossible to find and cross reference.
So instead just send it all right away - should only be a factor in rare cases of out of quota or close to out of space.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13131
Lustre-commit: 13b7cf4fabdd5 ("LU-13131 osc: Ensure immediate departure of sync write pages")
Reviewed-on: https://review.whamcloud.com/38453
--- fs/lustre/osc/osc_cache.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
fs/lustre/osc/osc_cache.c +16 -2
@@ -2015,7 +2015,6 @@ static unsigned int get_write_extents(struct osc_object *obj,while ((ext =list_first_entry_or_null (&obj->oo_hp_exts, struct osc_extent, oe_link))) { -LASSERT (ext->oe_state ==OES_CACHE );if (!try_to_add_extent_for_io (cli, ext, &data))return data.erd_page_count;EASSERT (ext->oe_nr_pages <= data.erd_max_pages, ext);
@@ -2701,7 +2700,22 @@ int osc_queue_sync_pages(const struct lu_env *env, const struct cl_io *io,/* Reuse the initial refcount for RPC, don't drop it */ osc_extent_state_set (ext,OES_LOCK_DONE );if (!ext->oe_rw) {/* write */ -list_add_tail (&ext->oe_link, &obj->oo_urgent_exts); +if (!ext->oe_srvlock && !ext->oe_dio) { +/* The most likely case here is from lack of grants +* so we are either out of quota or out of space. +* Since this means we are holding locks across +* potentially multi-striped IO, we must send out +* everything out instantly to avoid prolonged +* waits resulting in lock eviction (likely since +* the extended wait in osc_cache_enter() did not +* yield any additional grant due to a timeout. +* LU-13131 +*/ + ext->oe_hp =1 ; +list_add_tail (&ext->oe_link, &obj->oo_hp_exts); + }else { +list_add_tail (&ext->oe_link, &obj->oo_urgent_exts); + }osc_update_pending (obj,OBD_BRW_WRITE , page_count); }else {list_add_tail (&ext->oe_link, &obj->oo_reading_exts);
Signature · 1 line
1.8.3.1
PATCH 38/45 lnet: remove lnet_extract_iov()
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
The only place this is called, the src kvec is
NULL with length 0, so it returns 0.
So remove it.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13004
Lustre-commit: 571943f9cd090 ("LU-13004 lnet: remove lnet_extract_iov()")
Reviewed-on: https://review.whamcloud.com/37848
--- include/linux/lnet/lib-lnet.h | 4 ---- net/lnet/lnet/lib-move.c | 49 ------------------------------------------- 2 files changed, 53 deletions(-)
include/linux/lnet/lib-lnet.h +0 -4
@@ -614,10 +614,6 @@ int lnet_delay_rule_list(int pos, struct lnet_fault_attr *attr,void lnet_counters_reset (void );unsigned int lnet_iov_nob (unsigned int niov,struct kvec *iov); -int lnet_extract_iov (int dst_niov,struct kvec *dst, -int src_niov,const struct kvec *src, -unsigned int offset,unsigned int len); -unsigned int lnet_kiov_nob (unsigned int niov,struct bio_vec *iov); int lnet_extract_kiov(int dst_niov, struct bio_vec *dst, int src_niov, const struct bio_vec *src,
net/lnet/lnet/lib-move.c +0 -49
55 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 39/45 lnet: simplify ksock_tx.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
The tx_frags union in 'struct ksock_tx' is largely unnecessary. The payload is always lnet_kiov_t, the only kvec is a header. So replace the union with just those two fields.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13004
Lustre-commit: 28ffde29603b1 ("LU-13004 lnet: simplify ksock_tx.")
Reviewed-on: https://review.whamcloud.com/37850
--- net/lnet/klnds/socklnd/socklnd.h | 13 +++---------- net/lnet/klnds/socklnd/socklnd_cb.c | 8 ++++---- 2 files changed, 7 insertions(+), 14 deletions(-)
net/lnet/klnds/socklnd/socklnd.h +3 -10
@@ -291,18 +291,11 @@ struct ksock_tx { /* transmit packet */struct ksock_msg tx_msg;/* socklnd message buffer */ int tx_desc_size;/* size of this descriptor */ enum lnet_msg_hstatus tx_hstatus;/* health status of tx */ -union { -struct { -struct kvec iov;/* virt hdr */ -struct bio_vec kiov[0 ];/* paged payload */ - } paged; -struct { -struct kvec iov[1 ];/* virt hdr + payload */ - } virt; - } tx_frags; +struct kvec tx_hdr;/* virt hdr */ +struct bio_vec tx_payload[0 ];/* paged payload */ }; -#define KSOCK_NOOP_TX_SIZE (offsetof(struct ksock_tx, tx_frags.paged.kiov[0])) +#define KSOCK_NOOP_TX_SIZE (offsetof(struct ksock_tx, tx_payload[0]))/* network zero copy callback descriptor embedded in struct ksock_tx */
net/lnet/klnds/socklnd/socklnd_cb.c +4 -4
@@ -79,7 +79,7 @@ struct ksock_tx * tx->tx_lnetmsg =NULL ; tx->tx_kiov =NULL ; tx->tx_nkiov =0 ; - tx->tx_iov = tx->tx_frags.virt.iov; + tx->tx_iov = &tx->tx_hdr; tx->tx_niov =1 ; tx->tx_nonblk = nonblk;
@@ -933,7 +933,7 @@ struct ksock_route *LASSERT (!in_interrupt ()); desc_size = offsetof(struct ksock_tx , - tx_frags.paged.kiov[payload_niov]); + tx_payload[payload_niov]);if (lntmsg->msg_vmflush) mpflag =memalloc_noreclaim_save ();
@@ -950,8 +950,8 @@ struct ksock_route * tx->tx_lnetmsg = lntmsg; tx->tx_niov =1 ; - tx->tx_iov = &tx->tx_frags.paged.iov; - tx->tx_kiov = tx->tx_frags.paged.kiov; + tx->tx_iov = &tx->tx_hdr; + tx->tx_kiov = tx->tx_payload; tx->tx_nkiov =lnet_extract_kiov (payload_niov, tx->tx_kiov, payload_niov, payload_kiov, payload_offset, payload_nob);
Signature · 1 line
1.8.3.1
PATCH 40/45 lnet: socklnd: discard tx_iov.
In reply to James SimmonsFrom: Mr NeilBrown <[email protected]>
tx_iov always points to tx_hdr, so we can discard tx_iov, and just use &tx_hdr.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13004
Lustre-commit: 89637985b9897 ("LU-13004 socklnd: discard tx_iov.")
Reviewed-on: https://review.whamcloud.com/37851
--- net/lnet/klnds/socklnd/socklnd.h | 3 +-- net/lnet/klnds/socklnd/socklnd_cb.c | 33 ++++++++++++++------------------- net/lnet/klnds/socklnd/socklnd_lib.c | 32 +++++++++++++------------------- net/lnet/klnds/socklnd/socklnd_proto.c | 10 +++++----- 4 files changed, 33 insertions(+), 45 deletions(-)
net/lnet/klnds/socklnd/socklnd.h +1 -2
@@ -277,7 +277,6 @@ struct ksock_tx { /* transmit packet */int tx_nob;/* # packet bytes */ int tx_resid;/* residual bytes */ int tx_niov;/* # packet iovec frags */ -struct kvec *tx_iov;/* packet iovec frags */ int tx_nkiov;/* # packet page frags */ unsigned short tx_zc_aborted;/* aborted ZC request */ unsigned short tx_zc_capable:1;/* payload is large enough for ZC */
@@ -675,7 +674,7 @@ int ksocknal_recv_hello(struct lnet_ni *ni, struct ksock_conn *conn,void ksocknal_lib_push_conn (struct ksock_conn *conn);int ksocknal_lib_get_conn_addrs (struct ksock_conn *conn);int ksocknal_lib_setup_sock (struct socket *so); -int ksocknal_lib_send_iov (struct ksock_conn *conn,struct ksock_tx *tx); +int ksocknal_lib_send_hdr (struct ksock_conn *conn,struct ksock_tx *tx);int ksocknal_lib_send_kiov (struct ksock_conn *conn,struct ksock_tx *tx);void ksocknal_lib_eager_ack (struct ksock_conn *conn);int ksocknal_lib_recv (struct ksock_conn *conn);
net/lnet/klnds/socklnd/socklnd_cb.c +14 -19
@@ -79,7 +79,6 @@ struct ksock_tx * tx->tx_lnetmsg =NULL ; tx->tx_kiov =NULL ; tx->tx_nkiov =0 ; - tx->tx_iov = &tx->tx_hdr; tx->tx_niov =1 ; tx->tx_nonblk = nonblk;
@@ -109,16 +108,16 @@ struct ksock_tx * }static int -ksocknal_send_iov (struct ksock_conn *conn,struct ksock_tx *tx) +ksocknal_send_hdr (struct ksock_conn *conn,struct ksock_tx *tx) { -struct kvec *iov = tx->tx_iov; +struct kvec *iov = &tx->tx_hdr;int nob;int rc;LASSERT (tx->tx_niov >0 ); -/* Never touch tx->tx_iov inside ksocknal_lib_send_iov() */ - rc =ksocknal_lib_send_iov (conn, tx); +/* Never touch tx->tx_hdr inside ksocknal_lib_send_hdr() */ + rc =ksocknal_lib_send_hdr (conn, tx);if (rc <=0 )/* sent nothing? */ return rc;
@@ -128,19 +127,16 @@ struct ksock_tx * tx->tx_resid -= nob;/* "consume" iov */ -do { -LASSERT (tx->tx_niov >0 ); +LASSERT (tx->tx_niov ==1 ); -if (nob < (int )iov->iov_len) { - iov->iov_base = (void *)((char *)iov->iov_base + nob); - iov->iov_len -= nob; -return rc; - } +if (nob < (int )iov->iov_len) { + iov->iov_base += nob; + iov->iov_len -= nob; +return rc; + } - nob -= iov->iov_len; - tx->tx_iov = ++iov; - tx->tx_niov--; - }while (nob); +LASSERT (nob == iov->iov_len); + tx->tx_niov--;return rc; }
@@ -207,7 +203,7 @@ struct ksock_tx * ksocknal_data.ksnd_enomem_tx--; rc = -EAGAIN ; }else if (tx->tx_niov) { - rc =ksocknal_send_iov (conn, tx); + rc =ksocknal_send_hdr (conn, tx); }else { rc =ksocknal_send_kiov (conn, tx); }
@@ -694,7 +690,7 @@ struct ksock_conn ** We always expect at least 1 mapped fragment containing the * complete ksocknal message header. */ -LASSERT (lnet_iov_nob (tx->tx_niov, tx->tx_iov) + +LASSERT (lnet_iov_nob (tx->tx_niov, &tx->tx_hdr) +lnet_kiov_nob (tx->tx_nkiov, tx->tx_kiov) == (unsigned int )tx->tx_nob);LASSERT (tx->tx_niov >=1 );
@@ -950,7 +946,6 @@ struct ksock_route * tx->tx_lnetmsg = lntmsg; tx->tx_niov =1 ; - tx->tx_iov = &tx->tx_hdr; tx->tx_kiov = tx->tx_payload; tx->tx_nkiov = lnet_extract_kiov(payload_niov, tx->tx_kiov, payload_niov, payload_kiov,
net/lnet/klnds/socklnd/socklnd_lib.c +13 -19
@@ -73,11 +73,11 @@ }int -ksocknal_lib_send_iov (struct ksock_conn *conn,struct ksock_tx *tx) +ksocknal_lib_send_hdr (struct ksock_conn *conn,struct ksock_tx *tx) {struct msghdr msg = { .msg_flags =MSG_DONTWAIT };struct socket *sock = conn->ksnc_sock; -int nob, i; +int nob =0 ; if (*ksocknal_tunables.ksnd_enable_csum && /* checksum enabled */ conn->ksnc_proto == &ksocknal_protocol_v2x && /* V2.x connection */
@@ -85,15 +85,15 @@ !tx->tx_msg.ksm_csum) /* not checksummed */ksocknal_lib_csum_tx (tx); -for (nob = i =0 ; i < tx->tx_niov; i++) - nob += tx->tx_iov[i].iov_len; +if (tx->tx_niov) + nob += tx->tx_hdr.iov_len;if (!list_empty (&conn->ksnc_tx_queue) || nob < tx->tx_resid) msg.msg_flags |=MSG_MORE ;iov_iter_kvec (&msg.msg_iter,WRITE , - tx->tx_iov, tx->tx_niov, nob); + &tx->tx_hdr, tx->tx_niov, nob);return sock_sendmsg (sock, &msg); }
@@ -208,28 +208,22 @@ static int lustre_csum(struct kvec *v, void *context)u32 csum;void *base; -LASSERT (tx->tx_iov[0 ].iov_base == &tx->tx_msg); +LASSERT (tx->tx_hdr.iov_base == &tx->tx_msg);LASSERT (tx->tx_conn);LASSERT (tx->tx_conn->ksnc_proto == &ksocknal_protocol_v2x); tx->tx_msg.ksm_csum =0 ; - csum =crc32_le (~0 , tx->tx_iov[0 ].iov_base, - tx->tx_iov[0 ].iov_len); + csum =crc32_le (~0 , tx->tx_hdr.iov_base, + tx->tx_hdr.iov_len); -if (tx->tx_kiov) { -for (i =0 ; i < tx->tx_nkiov; i++) { - base =kmap (tx->tx_kiov[i].bv_page) + - tx->tx_kiov[i].bv_offset; +for (i =0 ; i < tx->tx_nkiov; i++) { + base =kmap (tx->tx_kiov[i].bv_page) + + tx->tx_kiov[i].bv_offset; - csum =crc32_le (csum, base, tx->tx_kiov[i].bv_len); + csum =crc32_le (csum, base, tx->tx_kiov[i].bv_len); -kunmap (tx->tx_kiov[i].bv_page); - } - }else { -for (i =1 ; i < tx->tx_niov; i++) - csum =crc32_le (csum, tx->tx_iov[i].iov_base, - tx->tx_iov[i].iov_len); +kunmap (tx->tx_kiov[i].bv_page); }if (*ksocknal_tunables.ksnd_inject_csum_error) {
net/lnet/klnds/socklnd/socklnd_proto.c +5 -5
@@ -711,8 +711,8 @@LASSERT (tx->tx_msg.ksm_type !=KSOCK_MSG_NOOP );LASSERT (tx->tx_lnetmsg); - tx->tx_iov[0 ].iov_base = &tx->tx_lnetmsg->msg_hdr; - tx->tx_iov[0 ].iov_len =sizeof (struct lnet_hdr ); + tx->tx_hdr.iov_base = &tx->tx_lnetmsg->msg_hdr; + tx->tx_hdr.iov_len =sizeof (struct lnet_hdr ); tx->tx_nob = tx->tx_lnetmsg->msg_len +sizeof (struct lnet_hdr ); tx->tx_resid = tx->tx_lnetmsg->msg_len +sizeof (struct lnet_hdr );
@@ -721,19 +721,19 @@static void ksocknal_pack_msg_v2 (struct ksock_tx *tx) { - tx->tx_iov[0 ].iov_base = &tx->tx_msg; + tx->tx_hdr.iov_base = &tx->tx_msg;if (tx->tx_lnetmsg) {LASSERT (tx->tx_msg.ksm_type !=KSOCK_MSG_NOOP ); tx->tx_msg.ksm_u.lnetmsg.ksnm_hdr = tx->tx_lnetmsg->msg_hdr; - tx->tx_iov[0 ].iov_len =sizeof (struct ksock_msg ); + tx->tx_hdr.iov_len =sizeof (struct ksock_msg ); tx->tx_nob =sizeof (struct ksock_msg ) + tx->tx_lnetmsg->msg_len; tx->tx_resid =sizeof (struct ksock_msg ) + tx->tx_lnetmsg->msg_len; }else {LASSERT (tx->tx_msg.ksm_type ==KSOCK_MSG_NOOP ); - tx->tx_iov[0].iov_len = offsetof(struct ksock_msg, ksm_u.lnetmsg.ksnm_hdr); + tx->tx_hdr.iov_len = offsetof(struct ksock_msg, ksm_u.lnetmsg.ksnm_hdr); tx->tx_nob = offsetof(struct ksock_msg, ksm_u.lnetmsg.ksnm_hdr); tx->tx_resid = offsetof(struct ksock_msg, ksm_u.lnetmsg.ksnm_hdr); }
Signature · 1 line
1.8.3.1
PATCH 41/45 lustre: lmv: do not print MDTs that are inactive
In reply to James SimmonsFrom: Andreas Dilger <[email protected]>
Have lmv return -EAGAIN instead of -ENODATA for unconfigured MDTs. That avoids "lfs df -v" from printing a long list of invalid MDTs when trying to get the target state for non-rotational devices.
Add test for "lfs df -v" printing nonrotational state, as well as limiting the reported OST and MDT to configured devices.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13427
Lustre-commit: 3728c2825aa27 ("LU-13427 lmv: do not print MDTs that are inactive")
Reviewed-on: https://review.whamcloud.com/38165
--- fs/lustre/lmv/lmv_obd.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
fs/lustre/lmv/lmv_obd.c +4 -1
@@ -820,7 +820,10 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp,return -ENODEV ; tgt =lmv_tgt (lmv, index); -if (!tgt || !tgt->ltd_active) +if (!tgt) +return -EAGAIN ; + +if (!tgt->ltd_active)return -ENODATA ; mdc_obd =class_exp2obd (tgt->ltd_exp);
Signature · 1 line
1.8.3.1
PATCH 42/45 lnet: use the same src nid for discovery
In reply to James SimmonsFrom: Amir Shehata <[email protected]>
When discovering a remote peer (not on the same network) a GET is sent to the peer to retrieve the peer's interfaces. This is followed by a PUSH, if discovery is on, to push the node's interfaces However, if both node and peer have multiple interfaces it is likely that the GET and the PUSH will originate on different interfaces. When the peer receives the PUSH it will not be able to connect the two NIDs and will not be able to consolidate the node's NIDs. This issue is specific for remote peers because at the time the push handler is invoked the remote lpni has not been created yet. lnet_parse() creates the lpni of the gateway.
Similar to the strategy already in place of using the same source NID for all the messages of an RPC, discovery should use the same source NID for both the GET and PUSH.
This patch stores the source NID interfaces the GET was sent on and uses it for the PUSH.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13471
Lustre-commit: 71ca66bcd9c3a ("LU-13471 lnet: use the same src nid for discovery")
Reviewed-on: https://review.whamcloud.com/38320
--- include/linux/lnet/lib-types.h | 3 +++ net/lnet/lnet/peer.c | 11 ++++++++++- 2 files changed, 13 insertions(+), 1 deletion(-)
include/linux/lnet/lib-types.h +3 -0
@@ -578,6 +578,9 @@ struct lnet_peer {/* primary NID of the peer */ lnet_nid_t lp_primary_nid; +/* source NID to use during discovery */ +lnet_nid_t lp_disc_src_nid; +/* net to perform discovery on */ u32 lp_disc_net_id;
net/lnet/lnet/peer.c +10 -1
@@ -216,6 +216,7 @@init_waitqueue_head (&lp->lp_dc_waitq);spin_lock_init (&lp->lp_lock); lp->lp_primary_nid = nid; + lp->lp_disc_src_nid =LNET_NID_ANY ;if (lnet_peers_start_down ()) lp->lp_alive = false;else
@@ -2271,6 +2272,8 @@ static void lnet_peer_clear_discovery_error(struct lnet_peer *lp)spin_lock (&lp->lp_lock); + lp->lp_disc_src_nid = ev->target.nid; +/* * If some kind of error happened the contents of message * cannot be used. Set PING_FAILED to trigger a retry.
@@ -3088,9 +3091,15 @@ static int lnet_peer_send_push(struct lnet_peer *lp) goto fail_unlink; } - rc =LNetPut (LNET_NID_ANY , lp->lp_push_mdh, + rc =LNetPut (lp->lp_disc_src_nid, lp->lp_push_mdh,LNET_ACK_REQ , id,LNET_RESERVED_PORTAL ,LNET_PROTO_PING_MATCHBITS ,0 ,0 ); +/* reset the discovery nid. There is no need to restrict sending +* from that source, if we call lnet_push_update_to_peers(). It'll +* get set to a specific NID, if we initiate discovery from the +* scratch +*/ + lp->lp_disc_src_nid =LNET_NID_ANY ;if (rc) goto fail_unlink;
Signature · 1 line
1.8.3.1
PATCH 43/45 lustre: llite: check if page truncated in ll_write_begin()
In reply to James SimmonsFrom: Wang Shilong <[email protected]>
See following function flows:
CPU0 CPU1:
|->grab_cache_page_nowait
|->find_get_page
|->__find_get_page (page unlocked)
|->truncate page
|->trylock_page -->page might has been truncated afterSo it is possible that page might has been truncated after grab_cache_page_nowait() return even page lock is held.
We need check wheather vmpage->mapping change in ll_write_begin() otherwise, we will have truncated page with NULL mapping, which will trigger assertions in vvp_set_pagevec_dirty().
This patch also fix assertion string doesn't end in newline.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13493
Lustre-commit: 985de582849df ("LU-13493 llite: check if page truncated in ll_write_begin()")
Reviewed-on: https://review.whamcloud.com/38425
--- fs/lustre/llite/rw26.c | 16 +++++++++++++--- fs/lustre/llite/vvp_io.c | 2 +- 2 files changed, 14 insertions(+), 4 deletions(-)
fs/lustre/llite/rw26.c +13 -3
@@ -453,11 +453,12 @@ static int ll_prepare_partial_page(const struct lu_env *env, struct cl_io *io,return result; } -static int ll_tiny_write_begin (struct page *vmpage) +static int ll_tiny_write_begin (struct page *vmpage, +struct address_space *mapping) {/* Page must be present, up to date, dirty, and not in writeback. */ if (!vmpage || !PageUptodate (vmpage) || !PageDirty (vmpage) || -PageWriteback (vmpage)) +PageWriteback (vmpage) || vmpage->mapping != mapping)return -ENODATA ;return 0 ;
@@ -483,7 +484,7 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, lcc =ll_cl_find (file);if (!lcc) { vmpage =grab_cache_page_nowait (mapping, index); - result =ll_tiny_write_begin (vmpage); + result =ll_tiny_write_begin (vmpage, mapping); goto out; }
@@ -547,6 +548,15 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, } } +/* page was truncated */ +if (mapping != vmpage->mapping) { +CDEBUG (D_VFSTRACE ,"page: %lu was truncated\n" , index); +unlock_page (vmpage); +put_page (vmpage); + vmpage =NULL ; + goto again; + } + page =cl_page_find (env, clob, vmpage->index, vmpage,CPT_CACHEABLE );if (IS_ERR (page)) { result =PTR_ERR (page);
fs/lustre/llite/vvp_io.c +1 -1
@@ -899,7 +899,7 @@ void vvp_set_pagevec_dirty(struct pagevec *pvec)ClearPageReclaim (pvec->pages[i]);LASSERTF (page->mapping, -"mapping must be set. page %p, page->private (cl_page) %p" , +"mapping must be set. page %p, page->private (cl_page) %p\n" , page, (void *) page->private);/* Rest of code derived from __set_page_dirty_nobuffers */
Signature · 1 line
1.8.3.1
PATCH 44/45 lustre: dne: improve temp file name check
In reply to James SimmonsFrom: Lai Siyao <[email protected]>
Previously if all but two characters in file name suffix are digit, it's not treated as temp file, as is too strict if suffix length is short, e.g. 6. Change it to allow one character, and this non-digit character should not be the starting character.
WC-bug-id: https://jira.whamcloud.com/browse/LU-13481
Lustre-commit: 51e5749683838 ("LU-13481 dne: improve temp file name check")
Reviewed-on: https://review.whamcloud.com/38539
--- fs/lustre/include/lu_object.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/lustre/include/lu_object.h +2 -1
@@ -1309,7 +1309,8 @@ static inline bool lu_name_is_temp_file(const char *name, int namelen,* About 0.07% of randomly-generated names will slip through, * but this avoids 99.93% of cross-MDT renames for those files. */ -if (digit >= suffixlen -2 || upper == suffixlen || lower == suffixlen) +if ((digit >= suffixlen -1 && !isdigit (name[namelen - suffixlen])) || + upper == suffixlen || lower == suffixlen)return false;return true;
Signature · 1 line
1.8.3.1
PATCH 45/45 lustre: all: Cleanup LASSERTF uses missing newlines
In reply to James SimmonsFrom: Shaun Tancheff <[email protected]>
LASSERTF() usage that does not include a terminating newline present an unclear syslog entry from admins.
This adds the terminating newline for a few cases.
HPE-bug-id: LUS-8853
WC-bug-id: https://jira.whamcloud.com/browse/LU-13539
Lustre-commit: 74f8e405b2c9d ("LU-13539 all: Cleanup LASSERTF uses missing newlines")
Reviewed-on: https://review.whamcloud.com/38540
--- fs/lustre/lov/lov_cl_internal.h | 6 ++++-- fs/lustre/ptlrpc/client.c | 2 +- fs/lustre/ptlrpc/import.c | 2 +- 3 files changed, 6 insertions(+), 4 deletions(-)
fs/lustre/lov/lov_cl_internal.h +4 -2
@@ -342,7 +342,8 @@ static inline struct lov_layout_raid0 *lov_r0(struct lov_object *lov, int i) {LASSERT (lov->lo_type ==LLT_COMP );LASSERTF (i < lov->u.composite.lo_entry_count, -"entry %d entry_count %d" , i, lov->u.composite.lo_entry_count); +"entry %d entry_count %d\n" , i, + lov->u.composite.lo_entry_count);return &lov->u.composite.lo_entries[i].lle_raid0; }
@@ -372,7 +373,8 @@ static inline struct lov_layout_entry *lov_entry(struct lov_object *lov, int i) {LASSERT (lov->lo_type ==LLT_COMP );LASSERTF (i < lov->u.composite.lo_entry_count, -"entry %d entry_count %d" , i, lov->u.composite.lo_entry_count); +"entry %d entry_count %d\n" , i, + lov->u.composite.lo_entry_count);return &lov->u.composite.lo_entries[i]; }
fs/lustre/ptlrpc/client.c +1 -1
@@ -848,7 +848,7 @@ struct ptlrpc_request *__ptlrpc_request_alloc(struct obd_import *imp,if (request) {ptlrpc_cli_req_init (request); -LASSERTF ((unsigned long )imp >0x1000 ,"%p" , imp); +LASSERTF ((unsigned long )imp >0x1000 ,"%p\n" , imp);LASSERT (imp !=LP_POISON );LASSERTF ((unsigned long )imp->imp_client >0x1000 ,"%p\n" , imp->imp_client);
fs/lustre/ptlrpc/import.c +1 -1
@@ -1495,7 +1495,7 @@ int ptlrpc_import_recovery_state_machine(struct obd_import *imp)"%s: This client was evicted by %.*s; in progress operations using this service will fail.\n" , imp->imp_obd->obd_name, target_len, target_start); -LASSERTF (!obd_lbug_on_eviction,"LBUG upon eviction" ); +LASSERTF (!obd_lbug_on_eviction,"LBUG upon eviction\n" ); } CDEBUG(D_HA, "evicted from %s@%s; invalidating\n", obd2cli_tgt(imp->imp_obd),
Signature · 1 line
1.8.3.1
cover lustre: merged OpenSFS client patches from April 30 to today
In reply to James SimmonsQuoted 4 lines · James Simmons
Merge the client side patches that landed to the OpenSFS tree since Apirl 30 to today. Include a fid missing patches as well and one patch to fix issues with 2 patches from the earlier 600+ patch set. Please review to ensure their correctness.
Hi James,
I tried applying these and hit lots of conflicts.
What tree are they on top of?
Thanks,
NeilBrown
Quoted 159 lines
Alexander Boyko (1):
lustre: llog: allow delete of zero size llogAmir Shehata (2):
lnet: restrict gateway selection
lnet: use the same src nid for discoveryAndreas Dilger (5):
lustre: llite: verify truncated xattr is handled
lustre: obd: fix printing of client connection UUID
lustre: llite: restorell_dcompare()
lustre: ldlm: use proper units for timeouts
lustre: lmv: do not print MDTs that are inactiveAndriy Skulysh (1):
lustre: ptlrpc: connect to MDT stucksBobi Jam (2):
lustre: lov:lov_io_sub_init()) ASSERTION
lustre: llite: do not flush COW pages from mappingChris Horn (2):
lnet: Introduce constant for the lolnd NID
lnet: Add MD options for response trackingJames Simmons (2):
lustre: cleanups and bug fixes
lnet: stop using struct timevalLai Siyao (3):
lustre: uapi: addOBD_CONNECT2_FIDMAP
lustre: dne: support directory restripe
lustre: dne: improve temp file name checkMr NeilBrown (13):
lnet: mergelnet_md_allocintolnet_md_build.
lnet: always put a page list into structlnet_libmd
lnet: discard kvec option fromlnet_libmd.
lnet: removemsg_iovfromlnet_msg.
lnet: o2iblnd: discardkiblnd_setup_rd_iov
lustre: Remove inappropriate uses ofBIT()macro.
lnet: libcfs: useBIT()macro where appropriate
lustre: llite: clean uppcc_layout_wait()
lustre: misc: declare static chars as const where possible.
lnet: usekmem_cache_zallocas appropriate.
lnet: removelnet_extract_iov()
lnet: simplifyksock_tx.
lnet: socklnd: discardtx_iov.NeilBrown (2):
lustre: fid: revertseq_client_rpcpatch.
lustre: fld: convertcache_flushfile toLPROC_SEQ_FOPSOleg Drokin (3):
lustre: ptlrpc: return proper write count fromping_store
lustre: osc: Do not wait for grants for too long
lustre: osc: Ensure immediate departure of sync write pagesQian Yingjin (1):
lustre: Send file creation time to clientsSebastien Buisson (2):
lustre: sec: check permissions for changelogs access
lustre: mgc: protect from NULL exp inmgc_enqueue()Sergey Cheremencev (1):
lustre: quota: quota pools for OSTsShaun Tancheff (1):
lustre: all: Cleanup LASSERTF uses missing newlinesSwapnil Pimpale (1):
lustre: fallocate: Implement fallocate preallocate operationWang Shilong (3):
lustre: llite: fix to make jobstats work for async ra
lustre: llite: fix possible divide zero inll_use_fast_io()
lustre: llite: check if page truncated inll_write_begin()fs/lustre/fid/fid_request.c | 12 +-- fs/lustre/fld/fld_cache.c | 40 -------- fs/lustre/fld/fld_internal.h | 7 -- fs/lustre/fld/lproc_fld.c | 31 +++--- fs/lustre/include/cl_object.h | 44 ++++++++- fs/lustre/include/lu_object.h | 16 +-- fs/lustre/include/lustre_dlm.h | 17 ++-- fs/lustre/include/lustre_import.h | 4 +- fs/lustre/include/lustre_lmv.h | 94 ++++++++++++++++-- fs/lustre/include/lustre_req_layout.h | 3 + fs/lustre/include/lustre_swab.h | 2 +- fs/lustre/include/obd_support.h | 5 +- fs/lustre/include/obd_target.h | 54 ++++++++++ fs/lustre/ldlm/ldlm_lockd.c | 3 +- fs/lustre/ldlm/ldlm_request.c | 1 + fs/lustre/ldlm/ldlm_resource.c | 11 ++- fs/lustre/llite/dcache.c | 56 ++++++++--- fs/lustre/llite/dir.c | 33 +++++-- fs/lustre/llite/file.c | 84 +++++++++++++++- fs/lustre/llite/lcommon_cl.c | 2 + fs/lustre/llite/llite_internal.h | 6 +- fs/lustre/llite/llite_lib.c | 6 +- fs/lustre/llite/llite_mmap.c | 2 +- fs/lustre/llite/lproc_llite.c | 19 ++-- fs/lustre/llite/namei.c | 5 +- fs/lustre/llite/pcc.c | 17 ++-- fs/lustre/llite/rw.c | 13 ++- fs/lustre/llite/rw26.c | 16 ++- fs/lustre/llite/statahead.c | 2 +- fs/lustre/llite/vvp_io.c | 19 ++-- fs/lustre/llite/vvp_object.c | 2 +- fs/lustre/llite/xattr.c | 15 ++- fs/lustre/lmv/lmv_obd.c | 66 +++++++++---- fs/lustre/lov/lov_cl_internal.h | 6 +- fs/lustre/lov/lov_io.c | 31 ++++-- fs/lustre/mdc/mdc_changelog.c | 1 + fs/lustre/mdc/mdc_request.c | 41 +++++--- fs/lustre/mgc/mgc_request.c | 11 ++- fs/lustre/obdclass/cl_page.c | 1 - fs/lustre/obdclass/class_obd.c | 35 +++---- fs/lustre/obdclass/llog.c | 9 +- fs/lustre/obdclass/llog_cat.c | 3 +- fs/lustre/obdclass/lprocfs_status.c | 8 +- fs/lustre/obdclass/obd_mount.c | 2 +- fs/lustre/osc/osc_cache.c | 33 ++++++- fs/lustre/osc/osc_internal.h | 3 + fs/lustre/osc/osc_io.c | 56 +++++++++-- fs/lustre/osc/osc_request.c | 70 ++++++++++++- fs/lustre/ptlrpc/client.c | 2 +- fs/lustre/ptlrpc/events.c | 2 +- fs/lustre/ptlrpc/import.c | 4 +- fs/lustre/ptlrpc/layout.c | 26 ++++- fs/lustre/ptlrpc/llog_client.c | 4 + fs/lustre/ptlrpc/lproc_ptlrpc.c | 4 +- fs/lustre/ptlrpc/pack_generic.c | 16 ++- fs/lustre/ptlrpc/ptlrpc_internal.h | 19 ++++ fs/lustre/ptlrpc/wiretest.c | 16 ++- include/linux/lnet/lib-lnet.h | 56 +---------- include/linux/lnet/lib-types.h | 15 +-- include/uapi/linux/lnet/lnet-types.h | 11 +++ include/uapi/linux/lnet/lnetst.h | 8 +- include/uapi/linux/lustre/lustre_idl.h | 28 ++++-- include/uapi/linux/lustre/lustre_user.h | 51 +++++++++- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 122 +++-------------------- net/lnet/klnds/socklnd/socklnd.h | 16 +-- net/lnet/klnds/socklnd/socklnd_cb.c | 68 +++++-------- net/lnet/klnds/socklnd/socklnd_lib.c | 32 +++--- net/lnet/klnds/socklnd/socklnd_proto.c | 10 +- net/lnet/libcfs/debug.c | 2 +- net/lnet/libcfs/libcfs_string.c | 4 +- net/lnet/libcfs/module.c | 17 ++-- net/lnet/lnet/api-ni.c | 16 +-- net/lnet/lnet/config.c | 16 +-- net/lnet/lnet/lib-md.c | 168 ++++++++++++++++---------------- net/lnet/lnet/lib-me.c | 2 +- net/lnet/lnet/lib-move.c | 167 ++++++++++++++----------------- net/lnet/lnet/lib-msg.c | 5 +- net/lnet/lnet/lib-socket.c | 34 ++----- net/lnet/lnet/lo.c | 17 +---
Quoted 11 lines
net/lnet/lnet/peer.c | 17 +++- net/lnet/lnet/router.c | 2 +- net/lnet/selftest/conctl.c | 1 - net/lnet/selftest/conrpc.c | 6 +- net/lnet/selftest/conrpc.h | 1 - net/lnet/selftest/console.h | 1 - 85 files changed, 1201 insertions(+), 802 deletions(-) create mode 100644 fs/lustre/include/obd_target.hSignature · 1 line
1.8.3.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20200529/b82d6062/attachment.sig>
cover lustre: merged OpenSFS client patches from April 30 to today
Quoted 9 lines
Quoted 5 lines
Merge the client side patches that landed to the OpenSFS tree since Apirl 30 to today. Include a fid missing patches as well and one patch to fix issues with 2 patches from the earlier 600+ patch set. Please review to ensure their correctness.
Hi James,
I tried applying these and hit lots of conflicts.
What tree are they on top of?
This is based off the top of my tree. Currently I'm using my tree as the OpenSFS version of the Linux client so I only add things that land. Your tree is more the development / prototype tree since it has not yet landed things. So most likely its the LNet LU-13004 work that is conflicting. I also run the patches by checkpatch which has the LNET comment change that wasn't in the OpenSFS branch until just recently; OpenSFS commit d3f1ceeb123da9603cd5ee3acbd4ee638a995f10. That should reduce the conflicts going forward. We are getting to the point in which both our trees are almost in sync ;-)
Quoted 175 lines
Thanks,
NeilBrownQuoted 170 lines
Alexander Boyko (1):
lustre: llog: allow delete of zero size llogAmir Shehata (2):
lnet: restrict gateway selection
lnet: use the same src nid for discoveryAndreas Dilger (5):
lustre: llite: verify truncated xattr is handled
lustre: obd: fix printing of client connection UUID
lustre: llite: restorell_dcompare()
lustre: ldlm: use proper units for timeouts
lustre: lmv: do not print MDTs that are inactiveAndriy Skulysh (1):
lustre: ptlrpc: connect to MDT stucksBobi Jam (2):
lustre: lov:lov_io_sub_init()) ASSERTION
lustre: llite: do not flush COW pages from mappingChris Horn (2):
lnet: Introduce constant for the lolnd NID
lnet: Add MD options for response trackingJames Simmons (2):
lustre: cleanups and bug fixes
lnet: stop using struct timevalLai Siyao (3):
lustre: uapi: addOBD_CONNECT2_FIDMAP
lustre: dne: support directory restripe
lustre: dne: improve temp file name checkMr NeilBrown (13):
lnet: mergelnet_md_allocintolnet_md_build.
lnet: always put a page list into structlnet_libmd
lnet: discard kvec option fromlnet_libmd.
lnet: removemsg_iovfromlnet_msg.
lnet: o2iblnd: discardkiblnd_setup_rd_iov
lustre: Remove inappropriate uses ofBIT()macro.
lnet: libcfs: useBIT()macro where appropriate
lustre: llite: clean uppcc_layout_wait()
lustre: misc: declare static chars as const where possible.
lnet: usekmem_cache_zallocas appropriate.
lnet: removelnet_extract_iov()
lnet: simplifyksock_tx.
lnet: socklnd: discardtx_iov.NeilBrown (2):
lustre: fid: revertseq_client_rpcpatch.
lustre: fld: convertcache_flushfile toLPROC_SEQ_FOPSOleg Drokin (3):
lustre: ptlrpc: return proper write count fromping_store
lustre: osc: Do not wait for grants for too long
lustre: osc: Ensure immediate departure of sync write pagesQian Yingjin (1):
lustre: Send file creation time to clientsSebastien Buisson (2):
lustre: sec: check permissions for changelogs access
lustre: mgc: protect from NULL exp inmgc_enqueue()Sergey Cheremencev (1):
lustre: quota: quota pools for OSTsShaun Tancheff (1):
lustre: all: Cleanup LASSERTF uses missing newlinesSwapnil Pimpale (1):
lustre: fallocate: Implement fallocate preallocate operationWang Shilong (3):
lustre: llite: fix to make jobstats work for async ra
lustre: llite: fix possible divide zero inll_use_fast_io()
lustre: llite: check if page truncated inll_write_begin()fs/lustre/fid/fid_request.c | 12 +-- fs/lustre/fld/fld_cache.c | 40 -------- fs/lustre/fld/fld_internal.h | 7 -- fs/lustre/fld/lproc_fld.c | 31 +++--- fs/lustre/include/cl_object.h | 44 ++++++++- fs/lustre/include/lu_object.h | 16 +-- fs/lustre/include/lustre_dlm.h | 17 ++-- fs/lustre/include/lustre_import.h | 4 +- fs/lustre/include/lustre_lmv.h | 94 ++++++++++++++++-- fs/lustre/include/lustre_req_layout.h | 3 + fs/lustre/include/lustre_swab.h | 2 +- fs/lustre/include/obd_support.h | 5 +- fs/lustre/include/obd_target.h | 54 ++++++++++ fs/lustre/ldlm/ldlm_lockd.c | 3 +- fs/lustre/ldlm/ldlm_request.c | 1 + fs/lustre/ldlm/ldlm_resource.c | 11 ++- fs/lustre/llite/dcache.c | 56 ++++++++--- fs/lustre/llite/dir.c | 33 +++++-- fs/lustre/llite/file.c | 84 +++++++++++++++- fs/lustre/llite/lcommon_cl.c | 2 + fs/lustre/llite/llite_internal.h | 6 +- fs/lustre/llite/llite_lib.c | 6 +- fs/lustre/llite/llite_mmap.c | 2 +- fs/lustre/llite/lproc_llite.c | 19 ++-- fs/lustre/llite/namei.c | 5 +- fs/lustre/llite/pcc.c | 17 ++-- fs/lustre/llite/rw.c | 13 ++- fs/lustre/llite/rw26.c | 16 ++- fs/lustre/llite/statahead.c | 2 +- fs/lustre/llite/vvp_io.c | 19 ++-- fs/lustre/llite/vvp_object.c | 2 +- fs/lustre/llite/xattr.c | 15 ++- fs/lustre/lmv/lmv_obd.c | 66 +++++++++---- fs/lustre/lov/lov_cl_internal.h | 6 +- fs/lustre/lov/lov_io.c | 31 ++++-- fs/lustre/mdc/mdc_changelog.c | 1 + fs/lustre/mdc/mdc_request.c | 41 +++++--- fs/lustre/mgc/mgc_request.c | 11 ++- fs/lustre/obdclass/cl_page.c | 1 - fs/lustre/obdclass/class_obd.c | 35 +++---- fs/lustre/obdclass/llog.c | 9 +- fs/lustre/obdclass/llog_cat.c | 3 +- fs/lustre/obdclass/lprocfs_status.c | 8 +- fs/lustre/obdclass/obd_mount.c | 2 +- fs/lustre/osc/osc_cache.c | 33 ++++++- fs/lustre/osc/osc_internal.h | 3 + fs/lustre/osc/osc_io.c | 56 +++++++++-- fs/lustre/osc/osc_request.c | 70 ++++++++++++- fs/lustre/ptlrpc/client.c | 2 +- fs/lustre/ptlrpc/events.c | 2 +- fs/lustre/ptlrpc/import.c | 4 +- fs/lustre/ptlrpc/layout.c | 26 ++++- fs/lustre/ptlrpc/llog_client.c | 4 + fs/lustre/ptlrpc/lproc_ptlrpc.c | 4 +- fs/lustre/ptlrpc/pack_generic.c | 16 ++- fs/lustre/ptlrpc/ptlrpc_internal.h | 19 ++++ fs/lustre/ptlrpc/wiretest.c | 16 ++- include/linux/lnet/lib-lnet.h | 56 +---------- include/linux/lnet/lib-types.h | 15 +-- include/uapi/linux/lnet/lnet-types.h | 11 +++ include/uapi/linux/lnet/lnetst.h | 8 +- include/uapi/linux/lustre/lustre_idl.h | 28 ++++-- include/uapi/linux/lustre/lustre_user.h | 51 +++++++++- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 122 +++-------------------- net/lnet/klnds/socklnd/socklnd.h | 16 +-- net/lnet/klnds/socklnd/socklnd_cb.c | 68 +++++-------- net/lnet/klnds/socklnd/socklnd_lib.c | 32 +++--- net/lnet/klnds/socklnd/socklnd_proto.c | 10 +- net/lnet/libcfs/debug.c | 2 +- net/lnet/libcfs/libcfs_string.c | 4 +- net/lnet/libcfs/module.c | 17 ++-- net/lnet/lnet/api-ni.c | 16 +-- net/lnet/lnet/config.c | 16 +-- net/lnet/lnet/lib-md.c | 168 ++++++++++++++++---------------- net/lnet/lnet/lib-me.c | 2 +- net/lnet/lnet/lib-move.c | 167 ++++++++++++++----------------- net/lnet/lnet/lib-msg.c | 5 +- net/lnet/lnet/lib-socket.c | 34 ++----- net/lnet/lnet/lo.c | 17 +--- net/lnet/lnet/peer.c | 17 +++- net/lnet/lnet/router.c | 2 +- net/lnet/selftest/conctl.c | 1 - net/lnet/selftest/conrpc.c | 6 +- net/lnet/selftest/conrpc.h | 1 - net/lnet/selftest/console.h | 1 - 85 files changed, 1201 insertions(+), 802 deletions(-) create mode 100644 fs/lustre/include/obd_target.hSignature · 1 line
1.8.3.1
cover lustre: merged OpenSFS client patches from April 30 to today
Quoted 9 lines · James Simmons
Quoted 8 lines
Quoted 4 lines
Merge the client side patches that landed to the OpenSFS tree since Apirl 30 to today. Include a fid missing patches as well and one patch to fix issues with 2 patches from the earlier 600+ patch set. Please review to ensure their correctness.
Hi James,
I tried applying these and hit lots of conflicts.
What tree are they on top of?
Quoted 9 lines
This is based off the top of my tree. Currently I'm using my tree as the OpenSFS version of the Linux client so I only add things that land. Your tree is more the development / prototype tree since it has not yet landed things. So most likely its the LNet LU-13004 work that is conflicting. I also run the patches by checkpatch which has the LNET comment change that wasn't in the OpenSFS branch until just recently; OpenSFS commit
d3f1ceeb123da9603cd5ee3acbd4ee638a995f10. That should reduce the conflicts going forward. We are getting to the point in which both our trees are almost in sync ;-)
I don't think it makes any sense to maintain two separate trees like this.
Would you like to just keep maintaining your tree, and I'll stop with mine?
If so, is your git tree published somewhere?
Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20200623/0d25b182/attachment.sig>
Degremont
cover lustre: merged OpenSFS client patches from April 30 to today
?Le 23/06/2020 06:11, ? lustre-devel au nom de NeilBrown ? <lustre-devel-bounces at lists.lustre.org au nom de [email protected]> a ?crit :
On Mon, Jun 01 2020, James Simmons wrote:
>> > Merge the client side patches that landed to the OpenSFS >> > tree since Apirl 30 to today. Include a fid missing patches >> > as well and one patch to fix issues with 2 patches from the >> > earlier 600+ patch set. Please review to ensure their correctness. >>
>> Hi James,
>> I tried applying these and hit lots of conflicts.
>> What tree are they on top of?
>
> This is based off the top of my tree. Currently I'm using my tree as the > OpenSFS version of the Linux client so I only add things that land. Your > tree is more the development / prototype tree since it has not yet landed > things. So most likely its the LNet LU-13004 work that is conflicting. > I also run the patches by checkpatch which has the LNET comment change > that wasn't in the OpenSFS branch until just recently; OpenSFS commit > d3f1ceeb123da9603cd5ee3acbd4ee638a995f10. That should reduce the conflicts > going forward. We are getting to the point in which both our trees are > almost in sync ;-)
I don't think it makes any sense to maintain two separate trees like this.
Would you like to just keep maintaining your tree, and I'll stop with mine?
If so, is your git tree published somewhere?
By the way, where's your tree available Neil? I'm checking your github account but not sure this is there.
Aur?lien
cover lustre: merged OpenSFS client patches from April 30 to today
Quoted 26 lines · Aurelien
?Le 23/06/2020 06:11, ? lustre-devel au nom de NeilBrown ? <lustre-devel-bounces at lists.lustre.org au nom de [email protected]> a ?crit :
On Mon, Jun 01 2020, James Simmons wrote:
>> > Merge the client side patches that landed to the OpenSFS >> > tree since Apirl 30 to today. Include a fid missing patches >> > as well and one patch to fix issues with 2 patches from the >> > earlier 600+ patch set. Please review to ensure their correctness. >>
>> Hi James,
>> I tried applying these and hit lots of conflicts.
>> What tree are they on top of?
>
> This is based off the top of my tree. Currently I'm using my tree as the > OpenSFS version of the Linux client so I only add things that land. Your > tree is more the development / prototype tree since it has not yet landed > things. So most likely its the LNet LU-13004 work that is conflicting. > I also run the patches by checkpatch which has the LNET comment change > that wasn't in the OpenSFS branch until just recently; OpenSFS commit >d3f1ceeb123da9603cd5ee3acbd4ee638a995f10. That should reduce the conflicts > going forward. We are getting to the point in which both our trees are > almost in sync ;-)I don't think it makes any sense to maintain two separate trees like this.
Quoted 8 lines
Would you like to just keep maintaining your tree, and I'll stop with mine?
If so, is your git tree published somewhere?
By the way, where's your tree available Neil? I'm checking your github account but not sure this is there.
In the "linux" repo I have quite a few branches.
"lustre/lustre" is the latest lustre code (as of late april).
https://github.com/neilbrown/linux/tree/lustre/lustre
NeilBrown
Quoted 2 lines
Aur?lien
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20200624/49158995/attachment.sig>
cover lustre: merged OpenSFS client patches from April 30 to today
Quoted 8 lines · NeilBrown
Quoted 3 lines
By the way, where's your tree available Neil? I'm checking your github account but not sure this is there.
In the "linux" repo I have quite a few branches.
"lustre/lustre" is the latest lustre code (as of late april).
Just FYI, I've updated this to Linux 5.7 and *almost* current master. There is one patch which causes all my testing to fail, so I've postponed that one until I understand the problem.
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20200703/5d6bb3ec/attachment.sig>
cover lustre: merged OpenSFS client patches from April 30 to today
In reply to NeilBrownQuoted 22 lines · James Simmons
Quoted 10 lines
Quoted 9 lines
Quoted 5 lines
Merge the client side patches that landed to the OpenSFS tree since Apirl 30 to today. Include a fid missing patches as well and one patch to fix issues with 2 patches from the earlier 600+ patch set. Please review to ensure their correctness.
Hi James,
I tried applying these and hit lots of conflicts.
What tree are they on top of?Quoted 9 lines
This is based off the top of my tree. Currently I'm using my tree as the OpenSFS version of the Linux client so I only add things that land. Your tree is more the development / prototype tree since it has not yet landed things. So most likely its the LNet LU-13004 work that is conflicting. I also run the patches by checkpatch which has the LNET comment change that wasn't in the OpenSFS branch until just recently; OpenSFS commit
d3f1ceeb123da9603cd5ee3acbd4ee638a995f10. That should reduce the conflicts going forward. We are getting to the point in which both our trees are almost in sync ;-)
Quoted 7 lines
I don't think it makes any sense to maintain two separate trees like this.
Would you like to just keep maintaining your tree, and I'll stop with mine?
If so, is your git tree published somewhere?
At this point its okay to have two separate trees since our trees have two different purposes. Mind you I do plan to retire my tree in the near future for your tree. My tree:
https://github.com/jasimmons1973/lustre
is a 1 to 1 OpenSFS port to the Linux client. Your tree has new prototype work; most has landed to the OpenSFS tree; which in turn ends up in my tree. Work is still outstanding in your tree that needs to be properly vetted and merged. Also I have noticed the work in your tree that has landed to OpenSFS is different. Sometimes the OpenSFS branch has improvements missing from the Linux client prototype patch and some times the reverse is true. So those issues need to be sorted out as well. Your tree is needed to work out these last bits. Once done a sweep will need to be done on your tree to patch everything up.
cover lustre: merged OpenSFS client patches from April 30 to today
Quoted 27 lines · James Simmons
Quoted 27 lines · James Simmons
Quoted 10 lines
Quoted 9 lines
Quoted 5 lines
Merge the client side patches that landed to the OpenSFS tree since Apirl 30 to today. Include a fid missing patches as well and one patch to fix issues with 2 patches from the earlier 600+ patch set. Please review to ensure their correctness.
Hi James,
I tried applying these and hit lots of conflicts.
What tree are they on top of?Quoted 9 lines
This is based off the top of my tree. Currently I'm using my tree as the OpenSFS version of the Linux client so I only add things that land. Your tree is more the development / prototype tree since it has not yet landed things. So most likely its the LNet LU-13004 work that is conflicting. I also run the patches by checkpatch which has the LNET comment change that wasn't in the OpenSFS branch until just recently; OpenSFS commit
d3f1ceeb123da9603cd5ee3acbd4ee638a995f10. That should reduce the conflicts going forward. We are getting to the point in which both our trees are almost in sync ;-)I don't think it makes any sense to maintain two separate trees like this.
Would you like to just keep maintaining your tree, and I'll stop with mine?
Quoted 7 lines
Quoted 1 line
If so, is your git tree published somewhere?
At this point its okay to have two separate trees since our trees have two different purposes. Mind you I do plan to retire my tree in the near future for your tree. My tree:
Thanks.
Quoted 10 lines
is a 1 to 1 OpenSFS port to the Linux client. Your tree has new prototype work; most has landed to the OpenSFS tree; which in turn ends up in my tree. Work is still outstanding in your tree that needs to be properly vetted and merged. Also I have noticed the work in your tree that has landed to OpenSFS is different. Sometimes the OpenSFS branch has improvements missing from the Linux client prototype patch and some times the reverse is true. So those issues need to be sorted out as well. Your tree is needed to work out these last bits. Once done a sweep will need to be done on your tree to patch everything up.
Maybe...
I'm beginning to wonder it might be best to base our eventual submission the Linux on the OpenSFS master branch.
i.e. get it to a state where we can write a tool that collects all the code needed for the client and places it in the linux tree at the appropriate place. Then we submit that as a series of commits.
The big value proposition here is that the code we submit is demonstrable close to the code that we test.
I don't know ... maybe maintaining the parallel tree is a good idea. I'm just not as sure as I once was. The testing is really a big deal.
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20200625/928d57c2/attachment.sig>