staging/lustre: minor cleanup and Intel Lustre sync
cover staging/lustre: minor cleanup and Intel Lustre sync
Hi Greg,
The first two of the patchset are minor cleanup of lustre code. And the rest of it are patches ported from Intel Lustre tree. Please help to queue them.
Thanks,
Tao
Alexander.Boyko (1):
staging/lustre/crypto: add crc32c module loading to libcfs
Andreas Dilger (2):
staging/lustre/ldlm: print FID in lvbo_init(), lvbo_update staging/lustre/mdt: duplicate link names in directory
Andriy Skulysh (1):
staging/lustre/ldlm: Fix flock deadlock detection race
Bruce Korb (1):
staging/lustre/dlmlock: compress out unused space
Bruno Faccini (1):
staging/lustre/mdc: Keep resend FLocks
Dmitry Eremin (2):
staging/lustre/llite: A not locked mutex can be unlocked. staging/lustre/ptlrpc: race in pinger (use-after-free situation)
Fan Yong (3):
staging/lustre/lfsck: LFSCK 1.5 technical debts (3) staging/lustre/llite: return valid fsid for statfs staging/lustre/scrub: purge inconsistenct objects after OI scrub
Hiroya Nozaki (1):
staging/lustre/ptlrpc: Race between start and stop service threads
Jinshan Xiong (1):
staging/lustre/layout: introduce new layout for released files
John L. Hammond (7):
staging/lustre/llite: use READ, WRITE around ll_rw_stats_tally() staging/lustre/llite: check alloc in ll_file_data_get, ll_dir_ioctl staging/lustre/llite: check ll_prep_md_op_data() using IS_ERR() staging/lustre/md: fix lu_ucred.c boilerplate staging/lustre/obdclass: use a dummy structure for lu_ref_link staging/lustre/llite: call simple_setattr() from ll_md_setattr() staging/lustre/procfs: return -ENOMEM from lprocfs_register()
Keith Mannthey (3):
staging/lustre/osc: Check return code for lu_kmem_init staging/lustre/llite: error of listxattr when buffer is small staging/lustre/style: removes obsolete EXPORT_SYMTAB macros v2
Li Wei (1):
staging/lustre/ptlrpc: Translate between host and network errnos
Liu Xuezhao (3):
staging/lustre/fid: prepare FID module for client server split staging/lustre/fld: prepare FLD module for client server split staging/lustre/obdclass: add obd_target.h
Nathaniel Clark (1):
staging/lustre/obdclass: be more careful processing server name
Ned Bass (1):
staging/lustre/lmv: fix duplicate directory entries
Oleg Drokin (3):
staging/lustre/ldlm: split client namespaces into active and inactive staging/lustre: Only wake up ldlm_poold as frequently as the check
interval
staging/lustre: Make quota namespace refcounting consistent
Patrick Farrell (1):
staging/lustre/llite: Anonymous dentry incorrectly identified as root
Peng Tao (3):
staging/lustre: remove bogus ifndef EXPORT_SYMBOL staging/lustre/obdclass: restore lu_ref staging/lustre/lnet: remove empty file lnet/lnet/api-errno.c
Sebastien Buisson (4):
staging/lustre: fix 'program hangs' errors staging/lustre: fix 'code maintainability' errors staging/lustre/build: fix 'data race condition' issues staging/lustre/build: fix 'data race condition' issues
Thomas Leibovici (2):
staging/lustre/lmv: support DNE with HSM.
staging/lustre/lmv: support DNE with HSM.
jcl (5):
staging/lustre/mdc: layout lock rpc must not take rpc_lock staging/lustre/mdt: add macros for fid string len staging/lustre/llapi: add user space method for lov_user_md staging/lustre/llite: force lvb_data update after layout change staging/lustre/mdt: HSM on disk actions record
wang di (1):
staging/lustre/llite: missing last bit in ll_have_md_lock
.../lustre/include/linux/libcfs/libcfs_prim.h | 4 - drivers/staging/lustre/lnet/lnet/Makefile | 2 +- drivers/staging/lustre/lnet/lnet/api-errno.c | 39 -- drivers/staging/lustre/lnet/selftest/rpc.c | 4 +- drivers/staging/lustre/lustre/Kconfig | 15 + drivers/staging/lustre/lustre/fid/Makefile | 2 +- drivers/staging/lustre/lustre/fid/fid_handler.c | 661 -------------------- drivers/staging/lustre/lustre/fid/fid_internal.h | 36 +- drivers/staging/lustre/lustre/fid/fid_lib.c | 14 +- drivers/staging/lustre/lustre/fid/fid_request.c | 133 ++-- drivers/staging/lustre/lustre/fid/fid_store.c | 259 -------- drivers/staging/lustre/lustre/fid/lproc_fid.c | 3 - drivers/staging/lustre/lustre/fld/Makefile | 2 +- drivers/staging/lustre/lustre/fld/fld_handler.c | 447 ------------- drivers/staging/lustre/lustre/fld/fld_index.c | 426 ------------- drivers/staging/lustre/lustre/fld/fld_internal.h | 29 - drivers/staging/lustre/lustre/fld/fld_request.c | 40 +- drivers/staging/lustre/lustre/fld/lproc_fld.c | 205 +----- drivers/staging/lustre/lustre/include/cl_object.h | 10 +- .../lustre/lustre/include/linux/lustre_fsfilt.h | 10 - .../staging/lustre/lustre/include/lprocfs_status.h | 39 -- drivers/staging/lustre/lustre/include/lu_object.h | 23 +- drivers/staging/lustre/lustre/include/lu_ref.h | 161 +++-- .../lustre/lustre/include/lustre/lustre_errno.h | 215 +++++++ .../lustre/lustre/include/lustre/lustre_idl.h | 82 ++- .../lustre/lustre/include/lustre/lustre_user.h | 12 + .../staging/lustre/lustre/include/lustre_disk.h | 56 +- drivers/staging/lustre/lustre/include/lustre_dlm.h | 202 +----- .../lustre/lustre/include/lustre_dlm_flags.h | 460 ++++++++++++++ drivers/staging/lustre/lustre/include/lustre_fid.h | 113 ++-- drivers/staging/lustre/lustre/include/lustre_fld.h | 41 -- .../staging/lustre/lustre/include/lustre_import.h | 2 + drivers/staging/lustre/lustre/include/lustre_mdc.h | 6 +- drivers/staging/lustre/lustre/include/lustre_mdt.h | 84 --- drivers/staging/lustre/lustre/include/lustre_net.h | 32 + drivers/staging/lustre/lustre/include/md_object.h | 5 - drivers/staging/lustre/lustre/include/obd.h | 168 +---- drivers/staging/lustre/lustre/include/obd_lov.h | 10 - .../staging/lustre/lustre/include/obd_support.h | 1 + drivers/staging/lustre/lustre/lclient/lcommon_cl.c | 2 +- drivers/staging/lustre/lustre/ldlm/l_lock.c | 8 +- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 18 +- drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 45 +- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 196 +++--- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 6 +- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 84 ++- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 13 +- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 60 +- drivers/staging/lustre/lustre/libcfs/libcfs_cpu.c | 3 - drivers/staging/lustre/lustre/libcfs/libcfs_lock.c | 3 - drivers/staging/lustre/lustre/libcfs/libcfs_mem.c | 3 - .../lustre/lustre/libcfs/linux/linux-crypto.c | 2 + drivers/staging/lustre/lustre/llite/dcache.c | 7 +- drivers/staging/lustre/lustre/llite/dir.c | 17 +- drivers/staging/lustre/lustre/llite/file.c | 96 +-- .../staging/lustre/lustre/llite/llite_internal.h | 2 + drivers/staging/lustre/lustre/llite/llite_lib.c | 14 +- drivers/staging/lustre/lustre/llite/llite_mmap.c | 36 +- drivers/staging/lustre/lustre/llite/llite_nfs.c | 16 + drivers/staging/lustre/lustre/llite/lproc_llite.c | 10 +- drivers/staging/lustre/lustre/llite/namei.c | 5 +- drivers/staging/lustre/lustre/llite/super25.c | 2 +- drivers/staging/lustre/lustre/llite/vvp_io.c | 14 +- drivers/staging/lustre/lustre/llite/xattr.c | 7 +- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 203 +++++- .../staging/lustre/lustre/lov/lov_cl_internal.h | 11 +- drivers/staging/lustre/lustre/lov/lov_ea.c | 23 +- drivers/staging/lustre/lustre/lov/lov_io.c | 34 + drivers/staging/lustre/lustre/lov/lov_obd.c | 8 +- drivers/staging/lustre/lustre/lov/lov_object.c | 73 ++- drivers/staging/lustre/lustre/lov/lov_pack.c | 19 +- drivers/staging/lustre/lustre/lvfs/lvfs_linux.c | 1 - drivers/staging/lustre/lustre/mdc/mdc_locks.c | 29 +- drivers/staging/lustre/lustre/mdc/mdc_request.c | 19 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 8 +- drivers/staging/lustre/lustre/obdclass/Makefile | 7 +- drivers/staging/lustre/lustre/obdclass/cl_io.c | 17 +- drivers/staging/lustre/lustre/obdclass/cl_lock.c | 6 +- drivers/staging/lustre/lustre/obdclass/cl_page.c | 5 +- .../lustre/lustre/obdclass/linux/linux-module.c | 9 +- drivers/staging/lustre/lustre/obdclass/llog_osd.c | 4 - drivers/staging/lustre/lustre/obdclass/llog_swab.c | 17 + .../lustre/lustre/obdclass/lprocfs_jobstats.c | 562 ----------------- .../lustre/lustre/obdclass/lprocfs_status.c | 28 +- drivers/staging/lustre/lustre/obdclass/lu_object.c | 25 +- drivers/staging/lustre/lustre/obdclass/lu_ref.c | 401 +++++++++++- drivers/staging/lustre/lustre/obdclass/lu_ucred.c | 8 +- .../staging/lustre/lustre/obdclass/obd_config.c | 11 +- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 59 +- .../staging/lustre/lustre/obdecho/echo_client.c | 1 + drivers/staging/lustre/lustre/osc/osc_cache.c | 8 +- drivers/staging/lustre/lustre/osc/osc_lock.c | 124 ++-- drivers/staging/lustre/lustre/osc/osc_request.c | 4 + drivers/staging/lustre/lustre/ptlrpc/Makefile | 1 + drivers/staging/lustre/lustre/ptlrpc/errno.c | 382 +++++++++++ .../staging/lustre/lustre/ptlrpc/gss/lproc_gss.c | 6 +- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 3 +- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 4 + drivers/staging/lustre/lustre/ptlrpc/pinger.c | 39 +- drivers/staging/lustre/lustre/ptlrpc/service.c | 16 +- 100 files changed, 3018 insertions(+), 3879 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/lnet/api-errno.c delete mode 100644 drivers/staging/lustre/lustre/fid/fid_handler.c delete mode 100644 drivers/staging/lustre/lustre/fid/fid_store.c delete mode 100644 drivers/staging/lustre/lustre/fld/fld_handler.c delete mode 100644 drivers/staging/lustre/lustre/fld/fld_index.c create mode 100644 drivers/staging/lustre/lustre/include/lustre/lustre_errno.h create mode 100644 drivers/staging/lustre/lustre/include/lustre_dlm_flags.h delete mode 100644 drivers/staging/lustre/lustre/include/lustre_mdt.h delete mode 100644 drivers/staging/lustre/lustre/obdclass/lprocfs_jobstats.c create mode 100644 drivers/staging/lustre/lustre/ptlrpc/errno.c
Signature · 1 line
1.7.9.5
PATCH 1/48 staging/lustre: remove bogus ifndef EXPORT_SYMBOL
--- .../lustre/include/linux/libcfs/libcfs_prim.h | 4 ---- 1 file changed, 4 deletions(-)
drivers/staging/lustre/include/linux/libcfs/libcfs_prim.h +0 -4
@@ -40,10 +40,6 @@#ifndef __LIBCFS_PRIM_H__#define __LIBCFS_PRIM_H__ -#ifndef EXPORT_SYMBOL -# define EXPORT_SYMBOL (s) -#endif -/* * Schedule */
Signature · 1 line
1.7.9.5
PATCH 2/48 staging/lustre/obdclass: restore lu_ref
In reply to Peng Taolu_ref.[ch] were mistakenly removed by coan2. Add them back and guard with CONFIG_LUSTRE_DEBUG_LU_REF_CHECK.
--- drivers/staging/lustre/lustre/Kconfig | 10 + drivers/staging/lustre/lustre/include/lu_ref.h | 124 +++++-- drivers/staging/lustre/lustre/obdclass/Makefile | 5 +- drivers/staging/lustre/lustre/obdclass/lu_ref.c | 403 ++++++++++++++++++++++- 4 files changed, 511 insertions(+), 31 deletions(-)
drivers/staging/lustre/lustre/Kconfig +10 -0
@@ -40,6 +40,16 @@ config LUSTRE_OBD_MAX_IOCTL_BUFFER If unsure, use default. +config LUSTRE_DEBUG_LU_REF_CHECK + bool "Enable Lustre DEBUG object refcount checks" + depends on LUSTRE_FS + default false + help + This option is mainly for debug purpose. It enables Lustre code to track + references between objects. + + If unsure, say N. + config LUSTRE_DEBUG_EXPENSIVE_CHECK bool "Enable Lustre DEBUG checks" depends on LUSTRE_FS
drivers/staging/lustre/lustre/include/lu_ref.h +97 -27
163 lines not on this page · raw message
drivers/staging/lustre/lustre/obdclass/Makefile +3 -2
@@ -6,8 +6,9 @@ obdclass-y := linux/linux-module.o linux/linux-obdo.o linux/linux-sysctl.o \ lprocfs_jobstats.o lustre_handles.o lustre_peer.o llog_osd.o \ local_storage.o statfs_pack.o obdo.o obd_config.o obd_mount.o\ mea.o lu_object.o dt_object.o capa.o cl_object.o \ - cl_page.o cl_lock.o cl_io.o lu_ref.o acl.o idmap.o \ - lu_ucred.o + cl_page.o cl_lock.o cl_io.o acl.o idmap.o lu_ucred.o + +obdclass-$(CONFIG_LUSTRE_DEBUG_LU_REF_CHECK) += lu_ref.o ccflags-y := -I$(src)/../include
drivers/staging/lustre/lustre/obdclass/lu_ref.c +401 -2
410 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Greg Kroah-Hartman
maintainerRe: [PATCH 02/48] staging/lustre/obdclass: restore lu_ref
Quoted 2 lines · Peng Tao
lu_ref.[ch] were mistakenly removed by coan2. Add them back and guard with
CONFIG_LUSTRE_DEBUG_LU_REF_CHECK.
Ick, why? Why not use the in-kernel infrastructure for this type of thig, _iff_ you really need it.
Why do you need this at all, isn't the reference code fully debugged by now, given that this has been running for years? I would only expect to see something like this in new code that is still being developed.
thanks,
greg k-h
Re: [PATCH 02/48] staging/lustre/obdclass: restore lu_ref
Quoted 10 lines · Greg Kroah-Hartman
Quoted 2 lines · Peng Tao
lu_ref.[ch] were mistakenly removed by coan2. Add them back and guard with
CONFIG_LUSTRE_DEBUG_LU_REF_CHECK.Ick, why? Why not use the in-kernel infrastructure for this type of thig, _iff_ you really need it.
Why do you need this at all, isn't the reference code fully debugged by now, given that this has been running for years? I would only expect to see something like this in new code that is still being developed.
Lustre is still being developed, so it is useful to be able to enable this debugging to catch any new problems that are introduced.
Cheers, Andreas
Signature · 4 lines
Andreas Dilger Lustre Software Architect Intel High Performance Data Division
Greg Kroah-Hartman
maintainerRe: [PATCH 02/48] staging/lustre/obdclass: restore lu_ref
Quoted 10 lines · Andreas
Quoted 7 lines · Greg Kroah-Hartman
Quoted 2 lines · Peng Tao
lu_ref.[ch] were mistakenly removed by coan2. Add them back and guard with
CONFIG_LUSTRE_DEBUG_LU_REF_CHECK.Ick, why? Why not use the in-kernel infrastructure for this type of thig, _iff_ you really need it.
Quoted 6 lines
Quoted 3 lines
Why do you need this at all, isn't the reference code fully debugged by now, given that this has been running for years? I would only expect to see something like this in new code that is still being developed.
Lustre is still being developed, so it is useful to be able to enable this debugging to catch any new problems that are introduced.
Ok, then use the in-kernel infrastructure for this type of thing, don't roll your own and add custom stuff that will later just have to be ripped out and converted to use the in-kernel stuff.
thanks,
greg k-h
Re: [PATCH 02/48] staging/lustre/obdclass: restore lu_ref
Quoted 15 lines · Greg Kroah-Hartman
Quoted 14 lines · Andreas
Quoted 10 lines · Greg Kroah-Hartman
Quoted 2 lines · Peng Tao
lu_ref.[ch] were mistakenly removed by coan2. Add them back and guard with
CONFIG_LUSTRE_DEBUG_LU_REF_CHECK.Ick, why? Why not use the in-kernel infrastructure for this type of thig, _iff_ you really need it.
Why do you need this at all, isn't the reference code fully debugged by now, given that this has been running for years? I would only expect to see something like this in new code that is still being developed.
Quoted 7 lines
Quoted 2 lines
Lustre is still being developed, so it is useful to be able to enable this debugging to catch any new problems that are introduced.
Ok, then use the in-kernel infrastructure for this type of thing, don't roll your own and add custom stuff that will later just have to be ripped out and converted to use the in-kernel stuff.
OK. Will drop this one.
Thanks,
Tao
PATCH 3/48 staging/lustre/llite: use READ, WRITE around ll_rw_stats_tally()
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
In vvp_io_write_start() the stats function ll_rw_stats_tally() was incorrectly called with a rw argument of 0. Correct this and use the macros READ and WRITE in and around ll_rw_stats_tally() for clarity.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3384
Lustre-change: http://review.whamcloud.com/6447
--- drivers/staging/lustre/lustre/llite/lproc_llite.c | 10 ++++++---- drivers/staging/lustre/lustre/llite/vvp_io.c | 4 ++-- 2 files changed, 8 insertions(+), 6 deletions(-)
drivers/staging/lustre/lustre/llite/lproc_llite.c +6 -4
@@ -1303,8 +1303,9 @@ static int ll_rw_offset_stats_seq_show(struct seq_file *seq, void *v)/* We stored the discontiguous offsets here; print them first */ for (i =0 ; i <LL_OFFSET_HIST_MAX ; i++) {if (offset[i].rw_pid !=0 ) - seq_printf(seq,"%3c %10d %14Lu %14Lu %17lu %17lu %14Lu", - offset[i].rw_op ? 'W' : 'R', + seq_printf(seq, + "%3c %10d %14Lu %14Lu %17lu %17lu %14Lu", + offset[i].rw_op == READ ? 'R' : 'W', offset[i].rw_pid, offset[i].rw_range_start, offset[i].rw_range_end,
@@ -1315,8 +1316,9 @@ static int ll_rw_offset_stats_seq_show(struct seq_file *seq, void *v)/* Then print the current offsets for each process */ for (i =0 ; i <LL_PROCESS_HIST_MAX ; i++) {if (process[i].rw_pid !=0 ) - seq_printf(seq,"%3c %10d %14Lu %14Lu %17lu %17lu %14Lu", - process[i].rw_op ? 'W' : 'R', + seq_printf(seq, + "%3c %10d %14Lu %14Lu %17lu %17lu %14Lu", + process[i].rw_op == READ ? 'R' : 'W', process[i].rw_pid, process[i].rw_range_start, process[i].rw_last_file_pos,
drivers/staging/lustre/lustre/llite/vvp_io.c +2 -2
@@ -525,7 +525,7 @@ out: io->ci_continue =0 ; io->ci_nob += result;ll_rw_stats_tally (ll_i2sbi (inode), current->pid, - cio->cui_fd, pos, result,0 ); + cio->cui_fd, pos, result,READ ); result =0 ; }return result;
@@ -580,7 +580,7 @@ static int vvp_io_write_start(const struct lu_env *env, io->ci_continue =0 ; io->ci_nob += result;ll_rw_stats_tally (ll_i2sbi (inode), current->pid, - cio->cui_fd, pos, result,0 ); + cio->cui_fd, pos, result,WRITE ); result =0 ; }RETURN (result);
Signature · 1 line
1.7.9.5
PATCH 4/48 staging/lustre/llite: missing last bit in ll_have_md_lock
In reply to Peng TaoFrom: wang di <[email protected]>
Missing the last bit during INODELOCK check in ll_have_md_lock.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3385
Lustre-change: http://review.whamcloud.com/6438
--- drivers/staging/lustre/lustre/llite/file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/lustre/lustre/llite/file.c +1 -1
@@ -2448,7 +2448,7 @@ int ll_have_md_lock(struct inode *inode, __u64 *bits, ldlm_mode_t l_req_mode) ldlm_lockname[mode]); flags =LDLM_FL_BLOCK_GRANTED |LDLM_FL_CBPENDING |LDLM_FL_TEST_LOCK ; -for (i =0 ; i <MDS_INODELOCK_MAXSHIFT && *bits !=0 ; i++) { +for (i =0 ; i <=MDS_INODELOCK_MAXSHIFT && *bits !=0 ; i++) { policy.l_inodebits.bits = *bits & (1 << i);if (policy.l_inodebits.bits ==0 )continue ;
Signature · 1 line
1.7.9.5
PATCH 5/48 staging/lustre: fix 'program hangs' errors
In reply to Peng TaoFrom: Sebastien Buisson <[email protected]>
Fix 'program hangs' defects found by Coverity version 6.5.1:
Missing unlock (LOCK)
Returning without unlocking.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3054
Lustre-change: http://review.whamcloud.com/5870
--- drivers/staging/lustre/lnet/selftest/rpc.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/staging/lustre/lnet/selftest/rpc.c +3 -1
@@ -661,8 +661,10 @@ srpc_finish_service(struct srpc_service *sv)cfs_percpt_for_each (scd, i, sv->sv_cpt_data) {spin_lock (&scd->scd_lock); -if (!swi_deschedule_workitem (&scd->scd_buf_wi)) +if (!swi_deschedule_workitem (&scd->scd_buf_wi)) { +spin_unlock (&scd->scd_lock);return 0 ; + }if (scd->scd_buf_nposted >0 ) { CDEBUG(D_NET, "waiting for %d posted buffers to unlink",
Signature · 1 line
1.7.9.5
PATCH 6/48 staging/lustre/llite: check alloc in ll_file_data_get, ll_dir_ioctl
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
In ll_file_data_get() and ll_dir_ioctl() return error on failed allocations.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2753
Lustre-change: http://review.whamcloud.com/5845
--- drivers/staging/lustre/lustre/llite/dir.c | 2 ++ drivers/staging/lustre/lustre/llite/file.c | 2 ++ 2 files changed, 4 insertions(+)
drivers/staging/lustre/lustre/llite/dir.c +2 -0
@@ -1561,6 +1561,8 @@ out_rmdir:RETURN (rc);OBD_ALLOC_LARGE (lmm, lmmsize); +if (lmm ==NULL ) +RETURN (-ENOMEM );if (copy_from_user (lmm, lum, lmmsize))GOTO (free_lmm, rc = -EFAULT );
drivers/staging/lustre/lustre/llite/file.c +2 -0
@@ -55,6 +55,8 @@ struct ll_file_data *ll_file_data_get(void)struct ll_file_data *fd;OBD_SLAB_ALLOC_PTR_GFP (fd, ll_file_data_slab, __GFP_IO); +if (fd ==NULL ) +return NULL ; fd->fd_write_failed = false;return fd; }
Signature · 1 line
1.7.9.5
PATCH 7/48 staging/lustre/llite: A not locked mutex can be unlocked.
In reply to Peng TaoFrom: Dmitry Eremin <[email protected]>
In case of memory pressure a not locked mutex can be unlocked in function ll_file_open(). This is not allowed and subsequent behavior is not defined.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3157
Lustre-change: http://review.whamcloud.com/6028
--- drivers/staging/lustre/lustre/llite/file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/staging/lustre/lustre/llite/file.c +1 -1
@@ -526,7 +526,7 @@ int ll_file_open(struct inode *inode, struct file *file) fd =ll_file_data_get ();if (fd ==NULL ) -GOTO (out_och_free, rc = -ENOMEM ); +GOTO (out_openerr, rc = -ENOMEM ); fd->fd_file = file;if (S_ISDIR (inode->i_mode)) {
Signature · 1 line
1.7.9.5
PATCH 8/48 staging/lustre/llite: check ll_prep_md_op_data() using IS_ERR()
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
In ll_file_ioctl() and ll_swap_layouts() check the result of ll_prep_md_op_data() using IS_ERR().
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3283
Lustre-change: http://review.whamcloud.com/6275
--- drivers/staging/lustre/lustre/llite/file.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
drivers/staging/lustre/lustre/llite/file.c +12 -12
@@ -1832,12 +1832,12 @@ static int ll_swap_layouts(struct file *file1, struct file *file2, rc = -ENOMEM ; op_data =ll_prep_md_op_data (NULL , llss->inode1, llss->inode2,NULL ,0 ,0 ,LUSTRE_OPC_ANY , &msl); -if (op_data !=NULL ) { - rc =obd_iocontrol (LL_IOC_LOV_SWAP_LAYOUTS , -ll_i2mdexp (llss->inode1), -sizeof (*op_data), op_data,NULL ); -ll_finish_md_op_data (op_data); - } +if (IS_ERR (op_data)) +GOTO (free, rc =PTR_ERR (op_data)); + + rc =obd_iocontrol (LL_IOC_LOV_SWAP_LAYOUTS ,ll_i2mdexp (llss->inode1), +sizeof (*op_data), op_data,NULL ); +ll_finish_md_op_data (op_data); putgl:if (gid !=0 ) {
@@ -2031,9 +2031,9 @@ long ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg) op_data =ll_prep_md_op_data (NULL , inode,NULL ,NULL ,0 ,0 ,LUSTRE_OPC_ANY , hus); -if (op_data ==NULL ) { +if (IS_ERR (op_data)) {OBD_FREE_PTR (hus); -RETURN (-ENOMEM ); +RETURN (PTR_ERR (op_data)); } rc = obd_iocontrol(cmd, ll_i2mdexp(inode), sizeof(*op_data),
@@ -2069,9 +2069,9 @@ long ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg) op_data =ll_prep_md_op_data (NULL , inode,NULL ,NULL ,0 ,0 ,LUSTRE_OPC_ANY , hss); -if (op_data ==NULL ) { +if (IS_ERR (op_data)) {OBD_FREE_PTR (hss); -RETURN (-ENOMEM ); +RETURN (PTR_ERR (op_data)); } rc = obd_iocontrol(cmd, ll_i2mdexp(inode), sizeof(*op_data),
@@ -2093,9 +2093,9 @@ long ll_file_ioctl(struct file *file, unsigned int cmd, unsigned long arg) op_data =ll_prep_md_op_data (NULL , inode,NULL ,NULL ,0 ,0 ,LUSTRE_OPC_ANY , hca); -if (op_data ==NULL ) { +if (IS_ERR (op_data)) {OBD_FREE_PTR (hca); -RETURN (-ENOMEM ); +RETURN (PTR_ERR (op_data)); } rc = obd_iocontrol(cmd, ll_i2mdexp(inode), sizeof(*op_data),
Signature · 1 line
1.7.9.5
PATCH 9/48 staging/lustre/ldlm: print FID in lvbo_init(), lvbo_update
In reply to Peng TaoFrom: Andreas Dilger <[email protected]>
Print the namespace and OBD device name, as well as the first two lock resource fields (typically the FID) if there is an error with loading the object from disk. This will be more important with FID-on-OST and also the MDS. Using fid_extract_from_res_name() isn't possible in the LDLM code, since the lock resource may not be a FID.
Make fid_extract_quota_resid() argument order and name consistent with other fid_*_res() functions, with FID first and resource second.
Fix a bug in ofd_lvbo_init() where NULL lvb is accessed on error.
Print FID in ofd_lvbo_update() CDEBUG() and CERROR() messages.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2193
Lustre-change: http://review.whamcloud.com/4501
--- .../lustre/lustre/include/lustre/lustre_idl.h | 4 +- drivers/staging/lustre/lustre/include/lustre_fid.h | 89 ++++++++++---------- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 5 +- 3 files changed, 50 insertions(+), 48 deletions(-)
drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +2 -2
@@ -911,7 +911,7 @@ static inline int lu_fid_cmp(const struct lu_fid *f0,__diff_normalize (fid_ver (f0),fid_ver (f1)); } -static inline void ostid_cpu_to_le (struct ost_id *src_oi, +static inline void ostid_cpu_to_le (const struct ost_id *src_oi,struct ost_id *dst_oi) {if (fid_seq_is_mdt0 (ostid_seq (src_oi))) {
@@ -922,7 +922,7 @@ static inline void ostid_cpu_to_le(struct ost_id *src_oi, } } -static inline void ostid_le_to_cpu (struct ost_id *src_oi, +static inline void ostid_le_to_cpu (const struct ost_id *src_oi,struct ost_id *dst_oi) {if (fid_seq_is_mdt0 (ostid_seq (src_oi))) {
drivers/staging/lustre/lustre/include/lustre_fid.h +45 -44
140 lines not on this page · raw message
drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +3 -2
@@ -1128,8 +1128,9 @@ ldlm_resource_get(struct ldlm_namespace *ns, struct ldlm_resource *parent,OBD_FAIL_TIMEOUT (OBD_FAIL_LDLM_CREATE_RESOURCE ,2 ); rc = ns->ns_lvbo->lvbo_init (res);if (rc <0 ) { -CERROR ("lvbo_init failed for resource " -LPU64 ": rc %d\n" , name->name[0 ], rc); +CERROR ("%s: lvbo_init failed for resource " LPX64 ":" +LPX64 ": rc = %d\n" , ns->ns_obd->obd_name, + name->name[0 ], name->name[1 ], rc);if (res->lr_lvb_data) {OBD_FREE (res->lr_lvb_data, res->lr_lvb_len); res->lr_lvb_data =NULL ;
Signature · 1 line
1.7.9.5
PATCH 10/48 staging/lustre/ptlrpc: race in pinger (use-after-free situation)
In reply to Peng TaoFrom: Dmitry Eremin <[email protected]>
The race is result of use-after-free situation:
~ ptlrpc_stop_pinger() ~ ptlrpc_pinger_main()
---------------------------------------------------------------thread_set_flags(SVC_STOPPING)cfs_waitq_signal(pinger_thread) ...
... thread_set_flags(SVC_STOPPED)l_wait_event(thread_is_stopped)OBD_FREE_PTR(pinger_thread)
... cfs_waitq_signal(pinger_thread)
---------------------------------------------------------------
The memory used by pinger_thread might have been freed and reallocated to something else, when ptlrpc_pinger_main() used it in cvs_waitq_signal().
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3032
Lustre-change: http://review.whamcloud.com/6040
--- drivers/staging/lustre/lustre/ptlrpc/pinger.c | 39 ++++++++++--------------- 1 file changed, 16 insertions(+), 23 deletions(-)
drivers/staging/lustre/lustre/ptlrpc/pinger.c +16 -23
@@ -369,7 +369,7 @@ static int ptlrpc_pinger_main(void *arg)return 0 ; } -static struct ptlrpc_thread *pinger_thread =NULL ; +static struct ptlrpc_thread pinger_thread;int ptlrpc_start_pinger (void ) {
@@ -377,29 +377,25 @@ int ptlrpc_start_pinger(void)int rc;ENTRY ; -if (pinger_thread !=NULL ) +if (!thread_is_init (&pinger_thread) && + !thread_is_stopped (&pinger_thread))RETURN (-EALREADY ); -OBD_ALLOC_PTR (pinger_thread); -if (pinger_thread ==NULL ) -RETURN (-ENOMEM ); -init_waitqueue_head (&pinger_thread->t_ctl_waitq); +init_waitqueue_head (&pinger_thread.t_ctl_waitq);init_waitqueue_head (&suspend_timeouts_waitq); -strcpy (pinger_thread->t_name,"ll_ping" ); +strcpy (pinger_thread.t_name,"ll_ping" );/* CLONE_VM and CLONE_FILES just avoid a needless copy, because we * just drop the VM and FILES in cfs_daemonize_ctxt() right away. */ rc =PTR_ERR (kthread_run (ptlrpc_pinger_main, - pinger_thread, pinger_thread->t_name)); + &pinger_thread, pinger_thread.t_name));if (IS_ERR_VALUE (rc)) {CERROR ("cannot start thread: %d\n" , rc); -OBD_FREE (pinger_thread,sizeof (*pinger_thread)); - pinger_thread =NULL ;RETURN (rc); } -l_wait_event (pinger_thread->t_ctl_waitq, -thread_is_running (pinger_thread), &lwi); +l_wait_event (pinger_thread.t_ctl_waitq, +thread_is_running (&pinger_thread), &lwi);if (suppress_pings) CWARN("Pings will be suppressed at the request of the "
@@ -419,20 +415,17 @@ int ptlrpc_stop_pinger(void)int rc =0 ;ENTRY ; -if (pinger_thread ==NULL ) +if (!thread_is_init (&pinger_thread) && + !thread_is_stopped (&pinger_thread))RETURN (-EALREADY );ptlrpc_pinger_remove_timeouts (); -mutex_lock (&pinger_mutex); -thread_set_flags (pinger_thread,SVC_STOPPING ); -wake_up (&pinger_thread->t_ctl_waitq); -mutex_unlock (&pinger_mutex); +thread_set_flags (&pinger_thread,SVC_STOPPING ); +wake_up (&pinger_thread.t_ctl_waitq); -l_wait_event (pinger_thread->t_ctl_waitq, -thread_is_stopped (pinger_thread), &lwi); +l_wait_event (pinger_thread.t_ctl_waitq, +thread_is_stopped (&pinger_thread), &lwi); -OBD_FREE_PTR (pinger_thread); - pinger_thread =NULL ;RETURN (rc); }
@@ -617,8 +610,8 @@ int ptlrpc_pinger_remove_timeouts(void)void ptlrpc_pinger_wake_up () { -thread_add_flags (pinger_thread,SVC_EVENT ); -wake_up (&pinger_thread->t_ctl_waitq); +thread_add_flags (&pinger_thread,SVC_EVENT ); +wake_up (&pinger_thread.t_ctl_waitq); }/* Ping evictor thread */
Signature · 1 line
1.7.9.5
PATCH 11/48 staging/lustre/ptlrpc: Translate between host and network errnos
In reply to Peng TaoFrom: Li Wei <[email protected]>
Lustre puts system errors (e.g., ENOTCONN) on wire as numbers essentially specific to senders' architectures. While this is fine for x86-only sites, where receivers share the same error number definition with senders, problems will arise, however, for sites involving multiple architectures with different error number definitions. For instance, an ENOTCONN reply from a sparc server will be put on wire as -57, which, for an x86 client, means EBADSLT instead.
To solve the problem, this patch defines a set of network errors for on-wire or on-disk uses. These errors correspond to a subset of the x86 system errors and share the same number definition, maintaining compatibility with existing x86 clients and servers.
Then, either error numbers could be translated at run time, or all host errors going on wire could be replaced with network errors in the code. This patch does the former by introducing both generic and field-specific translation routines and calling them at proper places, so that translations for existing fields are transparent. (Personally, I tend to think the latter way might be worthwhile, as it is more straightforward conceptually. Do we really need so many different errors? Should errors returned by kernel routines really be passed up and eventually put on wire? There could even be security implications in that.)
Thank Fujitsu for the original idea and their contributions that make this available upstream.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2743
Lustre-change: http://review.whamcloud.com/5577
--- drivers/staging/lustre/lustre/Kconfig | 5 + .../lustre/lustre/include/lustre/lustre_errno.h | 215 +++++++++++ .../lustre/lustre/include/lustre/lustre_idl.h | 2 + drivers/staging/lustre/lustre/include/lustre_net.h | 32 ++ drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 3 +- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 2 + drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 6 +- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 10 + drivers/staging/lustre/lustre/mdc/mdc_request.c | 1 + drivers/staging/lustre/lustre/osc/osc_request.c | 2 + drivers/staging/lustre/lustre/ptlrpc/Makefile | 1 + drivers/staging/lustre/lustre/ptlrpc/errno.c | 382 ++++++++++++++++++++ drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 3 +- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 3 + 14 files changed, 662 insertions(+), 5 deletions(-) create mode 100644 drivers/staging/lustre/lustre/include/lustre/lustre_errno.h create mode 100644 drivers/staging/lustre/lustre/ptlrpc/errno.c
drivers/staging/lustre/lustre/Kconfig +5 -0
@@ -59,3 +59,8 @@ config LUSTRE_DEBUG_EXPENSIVE_CHECK expensive checks that may have a performance impact. Use with caution. If unsure, say N. + +config LUSTRE_TRANSLATE_ERRNOS + bool + depends on LUSTRE_FS && !X86 + default true
drivers/staging/lustre/lustre/include/lustre/lustre_errno.h new file +215 -0
215 lines not on this page · raw message
drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +2 -0
@@ -98,6 +98,8 @@/* Defn's shared with user-space. */ #include <lustre/lustre_user.h> +#include <lustre/lustre_errno.h> +/* * GENERAL STUFF */
drivers/staging/lustre/lustre/include/lustre_net.h +32 -0
@@ -3166,6 +3166,38 @@ lustre_shrink_reply(struct ptlrpc_request *req, int segment, req->rq_replen =lustre_shrink_msg (req->rq_repmsg, segment, newlen, move_data); } + +#ifdef CONFIG_LUSTRE_TRANSLATE_ERRNOS + +static inline int ptlrpc_status_hton (int h) +{ +/* +* Positive errnos must be network errnos, such as LUSTRE_EDEADLK, +* ELDLM_LOCK_ABORTED, etc. +*/ +if (h <0 ) +return -lustre_errno_hton (-h); +else +return h; +} + +static inline int ptlrpc_status_ntoh (int n) +{ +/* +* See the comment in ptlrpc_status_hton(). +*/ +if (n <0 ) +return -lustre_errno_ntoh (-n); +else +return n; +} + +#else + +#define ptlrpc_status_hton (h) (h) +#define ptlrpc_status_ntoh (n) (n) + +#endif /** @} */ /** Change request phase of \a req to \a new_phase */
drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +2 -1
@@ -2208,7 +2208,8 @@ struct ldlm_resource *ldlm_lock_convert(struct ldlm_lock *lock, int new_mode,/* I can't check the type of lock here because the bitlock of lock * is not held here, so do the allocation blindly. -jay */ OBD_SLAB_ALLOC_PTR_GFP (node, ldlm_interval_slab, __GFP_IO); -if (node ==NULL )/* Actually, this causes EDEADLOCK to be returned */ +if (node ==NULL ) +/* Actually, this causes EDEADLOCK to be returned */ RETURN (NULL ); LASSERTF((new_mode == LCK_PW && lock->l_granted_mode == LCK_PR),
drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +2 -0
@@ -555,6 +555,8 @@ static int ldlm_handle_qc_callback(struct ptlrpc_request *req)RETURN (-EPROTO ); } + oqctl->qc_stat =ptlrpc_status_ntoh (oqctl->qc_stat); + cli->cl_qchk_stat = oqctl->qc_stat;return 0 ; }
drivers/staging/lustre/lustre/ldlm/ldlm_request.c +3 -3
@@ -1023,7 +1023,7 @@ static int ldlm_cli_convert_local(struct ldlm_lock *lock, int new_mode,ldlm_reprocess_all (res); rc =0 ; }else { - rc =EDEADLOCK ; + rc =LUSTRE_EDEADLK ; }LDLM_DEBUG (lock,"client-side local convert handler END" );LDLM_LOCK_PUT (lock);
@@ -1095,7 +1095,7 @@ int ldlm_cli_convert(struct lustre_handle *lockh, int new_mode, __u32 *flags)GOTO (out, rc); } }else { - rc =EDEADLOCK ; + rc =LUSTRE_EDEADLK ; }EXIT ; out:
@@ -1248,7 +1248,7 @@ int ldlm_cli_cancel_req(struct obd_export *exp, struct list_head *cancels, }else { rc =ptlrpc_queue_wait (req); } -if (rc ==ESTALE ) { +if (rc ==LUSTRE_ESTALE ) { CDEBUG(D_DLMTRACE, "client/server (nid %s) " "out of sync -- not fatal\n", libcfs_nid2str(req->rq_import->
drivers/staging/lustre/lustre/mdc/mdc_locks.c +10 -0
@@ -840,6 +840,9 @@ resend: lockrep =req_capsule_server_get (&req->rq_pill, &RMF_DLM_REP );LASSERT (lockrep !=NULL ); + lockrep->lock_policy_res2 = +ptlrpc_status_ntoh (lockrep->lock_policy_res2); +/* Retry the create infinitely when we get -EINPROGRESS from * server. This is required by the new quota design. */ if (it && it->it_op & IT_CREAT &&
@@ -1139,6 +1142,7 @@ static int mdc_intent_getattr_async_interpret(const struct lu_env *env,struct lookup_intent *it;struct lustre_handle *lockh;struct obd_device *obddev; +struct ldlm_reply *lockrep;__u64 flags =LDLM_FL_HAS_INTENT ;ENTRY ;
@@ -1159,6 +1163,12 @@ static int mdc_intent_getattr_async_interpret(const struct lu_env *env,GOTO (out, rc); } + lockrep =req_capsule_server_get (&req->rq_pill, &RMF_DLM_REP ); +LASSERT (lockrep !=NULL ); + + lockrep->lock_policy_res2 = +ptlrpc_status_ntoh (lockrep->lock_policy_res2); + rc =mdc_finish_enqueue (exp, req, einfo, it, lockh, rc);if (rc)GOTO (out, rc);
drivers/staging/lustre/lustre/mdc/mdc_request.c +1 -0
@@ -1189,6 +1189,7 @@ static int mdc_ioc_hsm_progress(struct obd_export *exp,GOTO (out, rc = -EPROTO ); *req_hpk = *hpk; + req_hpk->hpk_errval =lustre_errno_hton (hpk->hpk_errval);ptlrpc_request_set_replen (req);
drivers/staging/lustre/lustre/osc/osc_request.c +2 -0
@@ -2347,6 +2347,8 @@ static int osc_enqueue_fini(struct ptlrpc_request *req, struct ost_lvb *lvb, &RMF_DLM_REP);LASSERT (rep !=NULL ); + rep->lock_policy_res1 = +ptlrpc_status_ntoh (rep->lock_policy_res1);if (rep->lock_policy_res1) rc = rep->lock_policy_res1; }
drivers/staging/lustre/lustre/ptlrpc/Makefile +1 -0
@@ -16,6 +16,7 @@ ptlrpc_objs += sec.o sec_bulk.o sec_gc.o sec_config.o sec_lproc.o ptlrpc_objs += sec_null.o sec_plain.o nrs.o nrs_fifo.o ptlrpc-y := $(ldlm_objs) $(ptlrpc_objs) +ptlrpc-$(CONFIG_LUSTRE_TRANSLATE_ERRNOS) += errno.c obj-$(CONFIG_PTLRPC_GSS) += gss/
drivers/staging/lustre/lustre/ptlrpc/errno.c new file +382 -0
382 lines not on this page · raw message
drivers/staging/lustre/lustre/ptlrpc/niobuf.c +2 -1
@@ -400,7 +400,8 @@ int ptlrpc_send_reply(struct ptlrpc_request *req, int flags) req->rq_type =PTL_RPC_MSG_REPLY ;lustre_msg_set_type (req->rq_repmsg, req->rq_type); -lustre_msg_set_status (req->rq_repmsg, req->rq_status); +lustre_msg_set_status (req->rq_repmsg, +ptlrpc_status_hton (req->rq_status));lustre_msg_set_opc (req->rq_repmsg, req->rq_reqmsg ?lustre_msg_get_opc (req->rq_reqmsg) :0 );
drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +3 -0
@@ -642,6 +642,9 @@ static inline int lustre_unpack_ptlrpc_body_v2(struct ptlrpc_request *req,return -EINVAL ; } +if (!inout) + pb->pb_status =ptlrpc_status_ntoh (pb->pb_status); +return 0 ; }
Signature · 1 line
1.7.9.5
Re: [PATCH 11/48] staging/lustre/ptlrpc: Translate between host and network errnos
Quoted 62 lines · Peng Tao
From: Li Wei <[email protected]>
Lustre puts system errors (e.g., ENOTCONN) on wire as numbers essentially specific to senders' architectures. While this is fine for x86-only sites, where receivers share the same error number definition with senders, problems will arise, however, for sites involving multiple architectures with different error number definitions. For instance, an ENOTCONN reply from a sparc server will be put on wire as -57, which, for an x86 client, means EBADSLT instead.
To solve the problem, this patch defines a set of network errors for on-wire or on-disk uses. These errors correspond to a subset of the x86 system errors and share the same number definition, maintaining compatibility with existing x86 clients and servers.
Then, either error numbers could be translated at run time, or all host errors going on wire could be replaced with network errors in the code. This patch does the former by introducing both generic and field-specific translation routines and calling them at proper places, so that translations for existing fields are transparent. (Personally, I tend to think the latter way might be worthwhile, as it is more straightforward conceptually. Do we really need so many different errors? Should errors returned by kernel routines really be passed up and eventually put on wire? There could even be security implications in that.)
Thank Fujitsu for the original idea and their contributions that make this available upstream.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2743
Lustre-change: http://review.whamcloud.com/5577Signed-off-by Li Wei <[email protected]>Reviewed-by Andreas Dilger <[email protected]>Reviewed-by Hiroya Nozaki <[email protected]>Reviewed-by Oleg Drokin <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Andreas Dilger <[email protected]>--- drivers/staging/lustre/lustre/Kconfig | 5 + .../lustre/lustre/include/lustre/lustre_errno.h | 215 +++++++++++ .../lustre/lustre/include/lustre/lustre_idl.h | 2 + drivers/staging/lustre/lustre/include/lustre_net.h | 32 ++ drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 3 +- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 2 + drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 6 +- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 10 + drivers/staging/lustre/lustre/mdc/mdc_request.c | 1 + drivers/staging/lustre/lustre/osc/osc_request.c | 2 + drivers/staging/lustre/lustre/ptlrpc/Makefile | 1 + drivers/staging/lustre/lustre/ptlrpc/errno.c | 382 ++++++++++++++++++++ drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 3 +- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 3 + 14 files changed, 662 insertions(+), 5 deletions(-) create mode 100644 drivers/staging/lustre/lustre/include/lustre/lustre_errno.h create mode 100644 drivers/staging/lustre/lustre/ptlrpc/errno.cdrivers/staging/lustre/lustre/Kconfig +0
-0
drivers/staging/lustre/lustre/Kconfig config LUSTRE_DEBUG_EXPENSIVE_CHECK · line 66
Quoted 8 lines
drivers/staging/lustre/lustre/Kconfig +5
-0@@ -59,3 +59,8 @@ config LUSTRE_DEBUG_EXPENSIVE_CHECK expensive checks that may have a performance impact. Use with caution. If unsure, say N. + +config LUSTRE_TRANSLATE_ERRNOS + bool + depends on LUSTRE_FS && !X86 + default true
Did you mean "default y"?
Paul Bolle
Re: [PATCH 11/48] staging/lustre/ptlrpc: Translate between host and network errnos
Quoted 65 lines · Paul Bolle
Quoted 64 lines · Peng Tao
From: Li Wei <[email protected]>
Lustre puts system errors (e.g., ENOTCONN) on wire as numbers essentially specific to senders' architectures. While this is fine for x86-only sites, where receivers share the same error number definition with senders, problems will arise, however, for sites involving multiple architectures with different error number definitions. For instance, an ENOTCONN reply from a sparc server will be put on wire as -57, which, for an x86 client, means EBADSLT instead.
To solve the problem, this patch defines a set of network errors for on-wire or on-disk uses. These errors correspond to a subset of the x86 system errors and share the same number definition, maintaining compatibility with existing x86 clients and servers.
Then, either error numbers could be translated at run time, or all host errors going on wire could be replaced with network errors in the code. This patch does the former by introducing both generic and field-specific translation routines and calling them at proper places, so that translations for existing fields are transparent. (Personally, I tend to think the latter way might be worthwhile, as it is more straightforward conceptually. Do we really need so many different errors? Should errors returned by kernel routines really be passed up and eventually put on wire? There could even be security implications in that.)
Thank Fujitsu for the original idea and their contributions that make this available upstream.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2743
Lustre-change: http://review.whamcloud.com/5577Signed-off-by Li Wei <[email protected]>Reviewed-by Andreas Dilger <[email protected]>Reviewed-by Hiroya Nozaki <[email protected]>Reviewed-by Oleg Drokin <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Andreas Dilger <[email protected]>--- drivers/staging/lustre/lustre/Kconfig | 5 + .../lustre/lustre/include/lustre/lustre_errno.h | 215 +++++++++++ .../lustre/lustre/include/lustre/lustre_idl.h | 2 + drivers/staging/lustre/lustre/include/lustre_net.h | 32 ++ drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 3 +- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 2 + drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 6 +- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 10 + drivers/staging/lustre/lustre/mdc/mdc_request.c | 1 + drivers/staging/lustre/lustre/osc/osc_request.c | 2 + drivers/staging/lustre/lustre/ptlrpc/Makefile | 1 + drivers/staging/lustre/lustre/ptlrpc/errno.c | 382 ++++++++++++++++++++ drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 3 +- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 3 + 14 files changed, 662 insertions(+), 5 deletions(-) create mode 100644 drivers/staging/lustre/lustre/include/lustre/lustre_errno.h create mode 100644 drivers/staging/lustre/lustre/ptlrpc/errno.cdrivers/staging/lustre/lustre/Kconfig +0
-0@@ -59,3 +59,8 @@ config LUSTRE_DEBUG_EXPENSIVE_CHECK expensive checks that may have a performance impact.
Quoted 9 lines
drivers/staging/lustre/lustre/Kconfig config LUSTRE_DEBUG_EXPENSIVE_CHECK · line 66
Quoted 6 lines
drivers/staging/lustre/lustre/Kconfig +5
-0Use with caution. If unsure, say N. + +config LUSTRE_TRANSLATE_ERRNOS + bool + depends on LUSTRE_FS && !X86 + default trueDid you mean "default y"?
You are right. I wasn't aware of the mistake and have all the bool default to either "true" or "false"...
Thanks for pointing it out. I'll send patch to fix all of them.
Thanks,
Tao
PATCH 12/48 staging/lustre/mdc: layout lock rpc must not take rpc_lock
In reply to Peng TaoFrom: jcl <[email protected]>
When a client issue an RPC to get a layout lock, it must not hold rpc_lock because in case of a restore the rpc can be blocking for a long time
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3200
Lustre-change: http://review.whamcloud.com/6115
--- drivers/staging/lustre/lustre/include/lustre_mdc.h | 6 ++++-- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-)
drivers/staging/lustre/lustre/include/lustre_mdc.h +4 -2
@@ -86,7 +86,8 @@ static inline void mdc_get_rpc_lock(struct mdc_rpc_lock *lck, {ENTRY ; -if (it !=NULL && (it->it_op ==IT_GETATTR || it->it_op ==IT_LOOKUP )) +if (it !=NULL && (it->it_op ==IT_GETATTR || it->it_op ==IT_LOOKUP || + it->it_op ==IT_LAYOUT ))return ;/* This would normally block until the existing request finishes.
@@ -123,7 +124,8 @@ static inline void mdc_get_rpc_lock(struct mdc_rpc_lock *lck,static inline void mdc_put_rpc_lock (struct mdc_rpc_lock *lck,struct lookup_intent *it) { -if (it !=NULL && (it->it_op ==IT_GETATTR || it->it_op ==IT_LOOKUP )) +if (it !=NULL && (it->it_op ==IT_GETATTR || it->it_op ==IT_LOOKUP || + it->it_op ==IT_LAYOUT )) goto out;if (lck->rpcl_it ==MDC_FAKE_RPCL_IT ) {/* OBD_FAIL_MDC_RPCS_SEM */
drivers/staging/lustre/lustre/mdc/mdc_locks.c +1 -1
@@ -884,7 +884,7 @@ static int mdc_finish_intent_lock(struct obd_export *exp,struct mdt_body *mdt_body;struct ldlm_lock *lock;int rc; - +ENTRY ;LASSERT (request !=NULL );LASSERT (request !=LP_POISON );
Signature · 1 line
1.7.9.5
PATCH 13/48 staging/lustre/ldlm: split client namespaces into active and inactive
In reply to Peng TaoFrom: Oleg Drokin <[email protected]>
The main reason behind this is ldlm_poold walks all namespaces currently no matter if there are any locks or not. On large systems this could take quite a bit of time, esp. since ldlm_poold is currently woken up once per second.
Now every time a client namespace loses it's last resource it is placed into an inactive list that is not touched by ldlm_poold as pointless. On creation of a first resource in a namespace it is placed back into the active list.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2924
Lustre-change: http://review.whamcloud.com/5624
--- drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 - drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 46 +++++++++++++-- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 59 ++++++++++++++++---- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 55 +++++++++++++++--- 4 files changed, 135 insertions(+), 27 deletions(-)
drivers/staging/lustre/lustre/include/lustre_dlm.h +0 -2
@@ -1471,8 +1471,6 @@ void ldlm_namespace_free(struct ldlm_namespace *ns, struct obd_import *imp, int force);void ldlm_namespace_register (struct ldlm_namespace *ns,ldlm_side_t client);void ldlm_namespace_unregister (struct ldlm_namespace *ns,ldlm_side_t client); -void ldlm_namespace_move_locked (struct ldlm_namespace *ns,ldlm_side_t client); -struct ldlm_namespace *ldlm_namespace_first_locked (ldlm_side_t client);void ldlm_namespace_get (struct ldlm_namespace *ns);void ldlm_namespace_put (struct ldlm_namespace *ns);int ldlm_proc_setup (void );
drivers/staging/lustre/lustre/ldlm/ldlm_internal.h +40 -6
69 lines not on this page · raw message
drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +49 -10
126 lines not on this page · raw message
drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +46 -9
118 lines not on this page · raw message
Signature · 1 line
1.7.9.5
PATCH 14/48 staging/lustre: Only wake up ldlm_poold as frequently as the check interval
In reply to Peng TaoFrom: Oleg Drokin <[email protected]>
We used to wake up ldlm poold every second, but that's overkill, we should just see how much time is left until next closest recalc interval hits and sleep this much.
This will make "per-second" client grant statistic not actually per-second, but I don't think we need any precision in that code
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2924
Lustre-change: http://review.whamcloud.com/5793
--- drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 24 ++++++++++++-------- 2 files changed, 16 insertions(+), 10 deletions(-)
drivers/staging/lustre/lustre/include/lustre_dlm.h +1 -1
@@ -1643,7 +1643,7 @@ void unlock_res_and_lock(struct ldlm_lock *lock);* There are not used outside of ldlm. * @{ */ -void ldlm_pools_recalc (ldlm_side_t client); +int ldlm_pools_recalc (ldlm_side_t client);int ldlm_pools_init (void );void ldlm_pools_fini (void );
drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +15 -9
@@ -578,7 +578,6 @@ int ldlm_pool_recalc(struct ldlm_pool *pl) goto recalc;spin_lock (&pl->pl_lock); - recalc_interval_sec =cfs_time_current_sec () - pl->pl_recalc_time;if (recalc_interval_sec >0 ) {/* * Update pool statistics every 1s.
@@ -598,12 +597,12 @@ int ldlm_pool_recalc(struct ldlm_pool *pl) count = pl->pl_ops->po_recalc (pl);lprocfs_counter_add (pl->pl_stats,LDLM_POOL_RECALC_STAT , count); -return count; } + recalc_interval_sec = pl->pl_recalc_time -cfs_time_current_sec () + + pl->pl_recalc_period; -return 0 ; +return recalc_interval_sec; } -EXPORT_SYMBOL (ldlm_pool_recalc);/** * Pool shrink wrapper. Will call either client or server pool recalc callback
@@ -1144,12 +1143,13 @@ static int ldlm_pools_cli_shrink(SHRINKER_ARGS(sc, nr_to_scan, gfp_mask)) shrink_param(sc, gfp_mask)); } -void ldlm_pools_recalc (ldlm_side_t client) +int ldlm_pools_recalc (ldlm_side_t client) {__u32 nr_l =0 , nr_p =0 , l;struct ldlm_namespace *ns;struct ldlm_namespace *ns_old =NULL ;int nr, equal =0 ; +int time =50 ;/* seconds of sleep if no active namespaces */ /* * No need to setup pool limit for client pools.
@@ -1285,16 +1285,22 @@ void ldlm_pools_recalc(ldlm_side_t client)* After setup is done - recalc the pool. */ if (!skip) { -ldlm_pool_recalc (&ns->ns_pool); +int ttime =ldlm_pool_recalc (&ns->ns_pool); + +if (ttime < time) + time = ttime; +ldlm_namespace_put (ns); } } +return time; }EXPORT_SYMBOL (ldlm_pools_recalc);static int ldlm_pools_thread_main (void *arg) {struct ptlrpc_thread *thread = (struct ptlrpc_thread *)arg; +int s_time, c_time;ENTRY ;thread_set_flags (thread,SVC_RUNNING );
@@ -1309,14 +1315,14 @@ static int ldlm_pools_thread_main(void *arg)/* * Recal all pools on this tick. */ -ldlm_pools_recalc (LDLM_NAMESPACE_SERVER ); -ldlm_pools_recalc (LDLM_NAMESPACE_CLIENT ); + s_time =ldlm_pools_recalc (LDLM_NAMESPACE_SERVER ); + c_time =ldlm_pools_recalc (LDLM_NAMESPACE_CLIENT );/* * Wait until the next check time, or until we're * stopped. */ - lwi =LWI_TIMEOUT (cfs_time_seconds (LDLM_POOLS_THREAD_PERIOD ), + lwi =LWI_TIMEOUT (cfs_time_seconds (min (s_time, c_time)),NULL ,NULL ); l_wait_event(thread->t_ctl_waitq, thread_is_stopping(thread) ||
Signature · 1 line
1.7.9.5
PATCH 15/48 staging/lustre: Make quota namespace refcounting consistent
In reply to Peng TaoFrom: Oleg Drokin <[email protected]>
It seems quota namespace is needlessly referenced on connect, but that's not necessary as it could not go away until entire obd goes away.
On the other hand this extra reference disturbs other logic depending on empty namespace having zero refcount, so this patch drops such extra referencing.
This picks client side change of the original patch.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2924
Lustre-change: http://review.whamcloud.com/6234
--- drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 3 +-- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 6 +----- 2 files changed, 2 insertions(+), 7 deletions(-)
drivers/staging/lustre/lustre/ldlm/ldlm_internal.h +1 -2
@@ -84,8 +84,7 @@ static inline struct mutex *ldlm_namespace_lock(ldlm_side_t client) &ldlm_srv_namespace_lock : &ldlm_cli_namespace_lock; } -/* ns_bref is the number of resources in this namespace with the notable -* exception of quota namespaces which have their empty refcount at 1 */ +/* ns_bref is the number of resources in this namespace */ static inline int ldlm_ns_empty (struct ldlm_namespace *ns) {return atomic_read (&ns->ns_bref) ==0 ;
drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +1 -5
@@ -1171,11 +1171,7 @@ ldlm_resource_get(struct ldlm_namespace *ns, struct ldlm_resource *parent,/* Let's see if we happened to be the very first resource in this * namespace. If so, and this is a client namespace, we need to move * the namespace into the active namespaces list to be patrolled by -* the ldlm_poold. -* A notable exception, for quota namespaces qsd_lib.c already took a -* namespace reference, so it won't be participating in all of this, -* but I guess that's ok since we have no business cancelling quota -* locks anyway */ +* the ldlm_poold. */ if (ns_is_client (ns) && ns_refcount ==1 ) {mutex_lock (ldlm_namespace_lock (LDLM_NAMESPACE_CLIENT ));ldlm_namespace_move_to_active_locked (ns,LDLM_NAMESPACE_CLIENT );
Signature · 1 line
1.7.9.5
PATCH 16/48 staging/lustre/dlmlock: compress out unused space
In reply to Peng TaoFrom: Bruce Korb <[email protected]>
* lustre/include/lustre_dlm.h: Remove all bit fields and the unused weighing callback procedure. respell LDLM_AST_DISCARD_DATA as LDLM_FL_AST_DISCARD_DATA to match other flags.
* .gitignore: ignore emacs temporary files
* autogen.sh: rebuild the lock bits, if autogen is available.
* contrib/bit-masks/lustre_dlm_flags.def: define the ldlm_lock flags
* contrib/bit-masks/lustre_dlm_flags.tpl: template for emitting text
* contrib/bit-masks/Makefile: construct the .c and .h files The .c file is for constructing a crash extension and is not preserved.
* contrib/bit-masks/.gitignore: ignore built products
* lustre/contrib/wireshark/packet-lustre.c: use built files instead of local versions of the defines.
In the rest of the modified sources, replace flag field references with bit mask references.
* lustre/osc/osc_lock.c: removed osc_lock_weigh, too
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2771
Lustre-change: http://review.whamcloud.com/5312
--- drivers/staging/lustre/lustre/include/lustre_dlm.h | 198 +-------- .../lustre/lustre/include/lustre_dlm_flags.h | 460 ++++++++++++++++++++ drivers/staging/lustre/lustre/ldlm/l_lock.c | 8 +- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 48 +- drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c | 4 +- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 7 +- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 4 +- drivers/staging/lustre/lustre/llite/dir.c | 15 +- drivers/staging/lustre/lustre/llite/file.c | 19 +- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 9 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 8 +- drivers/staging/lustre/lustre/osc/osc_lock.c | 124 ++---- 13 files changed, 573 insertions(+), 333 deletions(-) create mode 100644 drivers/staging/lustre/lustre/include/lustre_dlm_flags.h
drivers/staging/lustre/lustre/include/lustre_dlm.h +5 -193
@@ -57,6 +57,8 @@#include <interval_tree.h> /* for interval_node{}, ldlm_extent */ #include <lu_ref.h> +#include "lustre_dlm_flags.h" +struct obd_ops ;struct obd_device ;
@@ -96,161 +98,6 @@ typedef enum { } ldlm_side_t;/** -* Declaration of flags sent through the wire. -**/ -#define LDLM_FL_LOCK_CHANGED 0x000001/* extent, mode, or resource changed */ - -/** -* If the server returns one of these flags, then the lock was put on that list. -* If the client sends one of these flags (during recovery ONLY!), it wants the -* lock added to the specified list, no questions asked. -*/ -#define LDLM_FL_BLOCK_GRANTED 0x000002 -#define LDLM_FL_BLOCK_CONV 0x000004 -#define LDLM_FL_BLOCK_WAIT 0x000008 - -/* Used to be LDLM_FL_CBPENDING 0x000010 moved to non-wire flags */ - -#define LDLM_FL_AST_SENT 0x000020/* blocking or cancel packet was -* queued for sending. */ -/* Used to be LDLM_FL_WAIT_NOREPROC 0x000040 moved to non-wire flags */ -/* Used to be LDLM_FL_CANCEL 0x000080 moved to non-wire flags */ - -/** -* Lock is being replayed. This could probably be implied by the fact that one -* of BLOCK_{GRANTED,CONV,WAIT} is set, but that is pretty dangerous. -*/ -#define LDLM_FL_REPLAY 0x000100 - -#define LDLM_FL_INTENT_ONLY 0x000200/* Don't grant lock, just do intent. */ - -/* Used to be LDLM_FL_LOCAL_ONLY 0x000400 moved to non-wire flags */ -/* Used to be LDLM_FL_FAILED 0x000800 moved to non-wire flags */ - -#define LDLM_FL_HAS_INTENT 0x001000/* lock request has intent */ - -/* Used to be LDLM_FL_CANCELING 0x002000 moved to non-wire flags */ -/* Used to be LDLM_FL_LOCAL 0x004000 moved to non-wire flags */ - -#define LDLM_FL_DISCARD_DATA 0x010000/* discard (no writeback) on cancel */ - -#define LDLM_FL_NO_TIMEOUT 0x020000/* Blocked by group lock - wait -* indefinitely */ - -/** file & record locking */ -#define LDLM_FL_BLOCK_NOWAIT 0x040000/* Server told not to wait if blocked. -* For AGL, OST will not send glimpse -* callback. */ -#define LDLM_FL_TEST_LOCK 0x080000 // return blocking lock - -/* Used to be LDLM_FL_LVB_READY 0x100000 moved to non-wire flags */ -/* Used to be LDLM_FL_KMS_IGNORE 0x200000 moved to non-wire flags */ -/* Used to be LDLM_FL_NO_LRU 0x400000 moved to non-wire flags */ - -/* Immediatelly cancel such locks when they block some other locks. Send -* cancel notification to original lock holder, but expect no reply. This is -* for clients (like liblustre) that cannot be expected to reliably response -* to blocking AST. */ -#define LDLM_FL_CANCEL_ON_BLOCK 0x800000 - -/* Flags flags inherited from parent lock when doing intents. */ -#define LDLM_INHERIT_FLAGS (LDLM_FL_CANCEL_ON_BLOCK) - -/* Used to be LDLM_FL_CP_REQD 0x1000000 moved to non-wire flags */ -/* Used to be LDLM_FL_CLEANED 0x2000000 moved to non-wire flags */ -/* Used to be LDLM_FL_ATOMIC_CB 0x4000000 moved to non-wire flags */ -/* Used to be LDLM_FL_BL_AST 0x10000000 moved to non-wire flags */ -/* Used to be LDLM_FL_BL_DONE 0x20000000 moved to non-wire flags */ - -/* measure lock contention and return -EUSERS if locking contention is high */ -#define LDLM_FL_DENY_ON_CONTENTION 0x40000000 - -/* These are flags that are mapped into the flags and ASTs of blocking locks */ -#define LDLM_AST_DISCARD_DATA 0x80000000/* Add FL_DISCARD to blocking ASTs */ - -/* Flags sent in AST lock_flags to be mapped into the receiving lock. */ -#define LDLM_AST_FLAGS (LDLM_FL_DISCARD_DATA) - -/* -* -------------------------------------------------------------------------- -* NOTE! Starting from this point, that is, LDLM_FL_* flags with values above -* 0x80000000 will not be sent over the wire. -* -------------------------------------------------------------------------- -*/ - -/** -* Declaration of flags not sent through the wire. -**/ - -/** -* Used for marking lock as a target for -EINTR while cp_ast sleep -* emulation + race with upcoming bl_ast. -*/ -#define LDLM_FL_FAIL_LOC 0x100000000ULL - -/** -* Used while processing the unused list to know that we have already -* handled this lock and decided to skip it. -*/ -#define LDLM_FL_SKIPPED 0x200000000ULL -/* this lock is being destroyed */ -#define LDLM_FL_CBPENDING 0x400000000ULL -/* not a real flag, not saved in lock */ -#define LDLM_FL_WAIT_NOREPROC 0x800000000ULL -/* cancellation callback already run */ -#define LDLM_FL_CANCEL 0x1000000000ULL -#define LDLM_FL_LOCAL_ONLY 0x2000000000ULL -/* don't run the cancel callback under ldlm_cli_cancel_unused */ -#define LDLM_FL_FAILED 0x4000000000ULL -/* lock cancel has already been sent */ -#define LDLM_FL_CANCELING 0x8000000000ULL -/* local lock (ie, no srv/cli split) */ -#define LDLM_FL_LOCAL 0x10000000000ULL -/* XXX FIXME: This is being added to b_size as a low-risk fix to the fact that -* the LVB filling happens _after_ the lock has been granted, so another thread -* can match it before the LVB has been updated. As a dirty hack, we set -* LDLM_FL_LVB_READY only after we've done the LVB poop. -* this is only needed on LOV/OSC now, where LVB is actually used and callers -* must set it in input flags. -* -* The proper fix is to do the granting inside of the completion AST, which can -* be replaced with a LVB-aware wrapping function for OSC locks. That change is -* pretty high-risk, though, and would need a lot more testing. */ -#define LDLM_FL_LVB_READY 0x20000000000ULL -/* A lock contributes to the known minimum size (KMS) calculation until it has -* finished the part of its cancelation that performs write back on its dirty -* pages. It can remain on the granted list during this whole time. Threads -* racing to update the KMS after performing their writeback need to know to -* exclude each other's locks from the calculation as they walk the granted -* list. */ -#define LDLM_FL_KMS_IGNORE 0x40000000000ULL -/* completion AST to be executed */ -#define LDLM_FL_CP_REQD 0x80000000000ULL -/* cleanup_resource has already handled the lock */ -#define LDLM_FL_CLEANED 0x100000000000ULL -/* optimization hint: LDLM can run blocking callback from current context -* w/o involving separate thread. in order to decrease cs rate */ -#define LDLM_FL_ATOMIC_CB 0x200000000000ULL - -/* It may happen that a client initiates two operations, e.g. unlink and -* mkdir, such that the server sends a blocking AST for conflicting -* locks to this client for the first operation, whereas the second -* operation has canceled this lock and is waiting for rpc_lock which is -* taken by the first operation. LDLM_FL_BL_AST is set by -* ldlm_callback_handler() in the lock to prevent the Early Lock Cancel -* (ELC) code from cancelling it. -* -* LDLM_FL_BL_DONE is to be set by ldlm_cancel_callback() when lock -* cache is dropped to let ldlm_callback_handler() return EINVAL to the -* server. It is used when ELC RPC is already prepared and is waiting -* for rpc_lock, too late to send a separate CANCEL RPC. */ -#define LDLM_FL_BL_AST 0x400000000000ULL -#define LDLM_FL_BL_DONE 0x800000000000ULL -/* Don't put lock into the LRU list, so that it is not canceled due to aging. -* Used by MGC locks, they are cancelled only at unmount or by callback. */ -#define LDLM_FL_NO_LRU 0x1000000000000ULL - -/** * The blocking callback is overloaded to perform two functions. These flags * indicate which operation should be performed. */
@@ -720,8 +567,6 @@ typedef int (*ldlm_completion_callback)(struct ldlm_lock *lock, __u64 flags, void *data);/** Type for glimpse callback function of a lock. */ typedef int (*ldlm_glimpse_callback )(struct ldlm_lock *lock,void *data); -/** Type for weight callback function of a lock. */ -typedef unsigned long (*ldlm_weigh_callback )(struct ldlm_lock *lock);/** Work list for sending GL ASTs to multiple locks. */ struct ldlm_glimpse_work {
@@ -890,9 +735,6 @@ struct ldlm_lock {*/ ldlm_glimpse_callback l_glimpse_ast; -/** XXX apparently unused "weight" handler. To be removed? */ -ldlm_weigh_callback l_weigh_ast; -/** * Lock export. * This is a pointer to actual client export for locks that were granted
@@ -919,11 +761,11 @@ struct ldlm_lock {ldlm_policy_data_t l_policy_data;/** -* Lock state flags. -* Like whenever we receive any blocking requests for this lock, etc. -* Protected by lr_lock. +* Lock state flags. Protected by lr_lock. +* \see lustre_dlm_flags.h where the bits are defined. */ __u64 l_flags; +/** * Lock r/w usage counters. * Protected by lr_lock.
@@ -952,34 +794,6 @@ struct ldlm_lock {/** Originally requested extent for the extent lock. */ struct ldlm_extent l_req_extent; - unsigned int l_failed:1, -/** -* Set for locks that were removed from class hash table and will be -* destroyed when last reference to them is released. Set by -* ldlm_lock_destroy_internal(). -* -* Protected by lock and resource locks. -*/ - l_destroyed:1, -/* -* it's set in lock_res_and_lock() and unset in unlock_res_and_lock(). -* -* NB: compared with check_res_locked(), checking this bit is cheaper. -* Also, spin_is_locked() is deprecated for kernel code; one reason is -* because it works only for SMP so user needs to add extra macros like -* LASSERT_SPIN_LOCKED for uniprocessor kernels. -*/ - l_res_locked:1, -/* -* It's set once we call ldlm_add_waiting_lock_res_locked() -* to start the lock-timeout timer and it will never be reset. -* -* Protected by lock_res_and_lock(). -*/ - l_waited:1, -/** Flag whether this is a server namespace lock. */ - l_ns_srv:1; -/* * Client-side-only members. */
@@ -1230,7 +1044,6 @@ struct ldlm_enqueue_info {void *ei_cb_bl;/** blocking lock callback */ void *ei_cb_cp;/** lock completion callback */ void *ei_cb_gl;/** lock glimpse callback */ -void *ei_cb_wg;/** lock weigh callback */ void *ei_cbdata;/** Data to be passed into callbacks. */ };
@@ -1328,7 +1141,6 @@ struct ldlm_callback_suite {ldlm_completion_callback lcs_completion;ldlm_blocking_callback lcs_blocking;ldlm_glimpse_callback lcs_glimpse; -ldlm_weigh_callback lcs_weigh; };/* ldlm_lockd.c */
drivers/staging/lustre/lustre/include/lustre_dlm_flags.h new file +460 -0
460 lines not on this page · raw message
drivers/staging/lustre/lustre/ldlm/l_lock.c +4 -4
@@ -51,12 +51,12 @@struct ldlm_resource *lock_res_and_lock (struct ldlm_lock *lock) {/* on server-side resource of lock doesn't change */ -if (!lock->l_ns_srv) +if ((lock->l_flags &LDLM_FL_NS_SRV ) ==0 )spin_lock (&lock->l_lock);lock_res (lock->l_resource); - lock->l_res_locked =1 ; + lock->l_flags |=LDLM_FL_RES_LOCKED ;return lock->l_resource; }EXPORT_SYMBOL (lock_res_and_lock);
@@ -67,10 +67,10 @@ EXPORT_SYMBOL(lock_res_and_lock);void unlock_res_and_lock (struct ldlm_lock *lock) {/* on server-side resource of lock doesn't change */ - lock->l_res_locked =0 ; + lock->l_flags &= ~LDLM_FL_RES_LOCKED ;unlock_res (lock->l_resource); -if (!lock->l_ns_srv) +if ((lock->l_flags &LDLM_FL_NS_SRV ) ==0 )spin_unlock (&lock->l_lock); }EXPORT_SYMBOL (unlock_res_and_lock);
drivers/staging/lustre/lustre/ldlm/ldlm_flock.c +1 -1
@@ -639,7 +639,7 @@ ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data) granted:OBD_FAIL_TIMEOUT (OBD_FAIL_LDLM_CP_CB_WAIT ,10 ); -if (lock->l_destroyed) { +if (lock->l_flags &LDLM_FL_DESTROYED ) {LDLM_DEBUG (lock,"client-side enqueue waking up: destroyed" );RETURN (0 ); }
drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +23 -25
164 lines not on this page · raw message
drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +2 -2
@@ -198,7 +198,7 @@ static void ldlm_handle_cp_callback(struct ptlrpc_request *req,schedule_timeout_and_set_state (TASK_INTERRUPTIBLE , to);if (lock->l_granted_mode == lock->l_req_mode || - lock->l_destroyed) + lock->l_flags &LDLM_FL_DESTROYED )break ; } }
@@ -238,7 +238,7 @@ static void ldlm_handle_cp_callback(struct ptlrpc_request *req, }lock_res_and_lock (lock); -if (lock->l_destroyed || +if ((lock->l_flags &LDLM_FL_DESTROYED ) || lock->l_granted_mode == lock->l_req_mode) {/* bug 11300: the lock has already been granted */ unlock_res_and_lock (lock);
drivers/staging/lustre/lustre/ldlm/ldlm_request.c +3 -4
@@ -160,7 +160,7 @@ static int ldlm_completion_tail(struct ldlm_lock *lock)long delay;int result; -if (lock->l_destroyed || lock->l_flags &LDLM_FL_FAILED ) { +if (lock->l_flags & (LDLM_FL_DESTROYED |LDLM_FL_FAILED )) {LDLM_DEBUG (lock,"client-side enqueue: destroyed" ); result = -EIO ; }else {
@@ -888,9 +888,8 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, }else {const struct ldlm_callback_suite cbs = { .lcs_completion = einfo->ei_cb_cp, - .lcs_blocking = einfo->ei_cb_bl, - .lcs_glimpse = einfo->ei_cb_gl, - .lcs_weigh = einfo->ei_cb_wg + .lcs_blocking = einfo->ei_cb_bl, + .lcs_glimpse = einfo->ei_cb_gl }; lock = ldlm_lock_create(ns, res_id, einfo->ei_type, einfo->ei_mode, &cbs, einfo->ei_cbdata,
drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +2 -2
@@ -1283,7 +1283,7 @@ void ldlm_resource_add_lock(struct ldlm_resource *res, struct list_head *head,LDLM_DEBUG (lock,"About to add this lock:\n" ); -if (lock->l_destroyed) { +if (lock->l_flags &LDLM_FL_DESTROYED ) {CDEBUG (D_OTHER ,"Lock destroyed, not adding to resource\n" );return ; }
@@ -1308,7 +1308,7 @@ void ldlm_resource_insert_lock_after(struct ldlm_lock *original,ldlm_resource_dump (D_INFO , res);LDLM_DEBUG (new,"About to insert this lock after %p:\n" , original); -if (new->l_destroyed) { +if (new->l_flags &LDLM_FL_DESTROYED ) {CDEBUG (D_OTHER ,"Lock destroyed, not adding to resource\n" ); goto out; }
drivers/staging/lustre/lustre/llite/dir.c +6 -9
@@ -356,15 +356,12 @@ struct page *ll_get_dir_page(struct inode *dir, __u64 hash, rc =md_lock_match (ll_i2sbi (dir)->ll_md_exp,LDLM_FL_BLOCK_GRANTED ,ll_inode2fid (dir),LDLM_IBITS , &policy, mode, &lockh);if (!rc) { -struct ldlm_enqueue_info einfo = {.ei_type =LDLM_IBITS , - .ei_mode = mode, - .ei_cb_bl = - ll_md_blocking_ast, - .ei_cb_cp = - ldlm_completion_ast, - .ei_cb_gl =NULL , - .ei_cb_wg =NULL , - .ei_cbdata =NULL }; +struct ldlm_enqueue_info einfo = { + .ei_type =LDLM_IBITS , + .ei_mode = mode, + .ei_cb_bl = ll_md_blocking_ast, + .ei_cb_cp = ldlm_completion_ast, + };struct lookup_intent it = { .it_op =IT_READDIR };struct ptlrpc_request *request;struct md_op_data *op_data;
drivers/staging/lustre/lustre/llite/file.c +11 -8
@@ -2290,9 +2290,11 @@ int ll_file_flock(struct file *file, int cmd, struct file_lock *file_lock) {struct inode *inode = file->f_dentry->d_inode;struct ll_sb_info *sbi =ll_i2sbi (inode); -struct ldlm_enqueue_info einfo = { .ei_type =LDLM_FLOCK , - .ei_cb_cp =ldlm_flock_completion_ast, - .ei_cbdata = file_lock }; +struct ldlm_enqueue_info einfo = { + .ei_type =LDLM_FLOCK , + .ei_cb_cp = ldlm_flock_completion_ast, + .ei_cbdata = file_lock, + };struct md_op_data *op_data;struct lustre_handle lockh = {0 };ldlm_policy_data_t flock = {{0 }};
@@ -3116,11 +3118,12 @@ int ll_layout_refresh(struct inode *inode, __u32 *gen)struct lookup_intent it;struct lustre_handle lockh;ldlm_mode_t mode; -struct ldlm_enqueue_info einfo = { .ei_type =LDLM_IBITS , - .ei_mode =LCK_CR , - .ei_cb_bl = ll_md_blocking_ast, - .ei_cb_cp = ldlm_completion_ast, - .ei_cbdata =NULL }; +struct ldlm_enqueue_info einfo = { + .ei_type =LDLM_IBITS , + .ei_mode =LCK_CR , + .ei_cb_bl = ll_md_blocking_ast, + .ei_cb_cp = ldlm_completion_ast, + };int rc;ENTRY ;
drivers/staging/lustre/lustre/mdc/mdc_locks.c +6 -3
@@ -1102,9 +1102,12 @@ int mdc_intent_lock(struct obd_export *exp, struct md_op_data *op_data,* this and use the request from revalidate. In this case, revalidate * never dropped its reference, so the refcounts are all OK */ if (!it_disposition (it,DISP_ENQ_COMPLETE )) { -struct ldlm_enqueue_info einfo = - {LDLM_IBITS ,it_to_lock_mode (it), cb_blocking, - ldlm_completion_ast,NULL ,NULL ,NULL }; +struct ldlm_enqueue_info einfo = { + .ei_type =LDLM_IBITS , + .ei_mode =it_to_lock_mode (it), + .ei_cb_bl = cb_blocking, + .ei_cb_cp = ldlm_completion_ast, + };/* For case if upper layer did not alloc fid, do it now. */ if (!fid_is_sane (&op_data->op_fid2) && it->it_op &IT_CREAT ) {
drivers/staging/lustre/lustre/mgc/mgc_request.c +6 -2
@@ -900,8 +900,12 @@ static int mgc_enqueue(struct obd_export *exp, struct lov_stripe_md *lsm, struct lustre_handle *lockh) {struct config_llog_data *cld = (struct config_llog_data *)data; -struct ldlm_enqueue_info einfo = { type, mode, mgc_blocking_ast, - ldlm_completion_ast,NULL ,NULL ,NULL }; +struct ldlm_enqueue_info einfo = { + .ei_type = type, + .ei_mode = mode, + .ei_cb_bl = mgc_blocking_ast, + .ei_cb_cp = ldlm_completion_ast, + };struct ptlrpc_request *req;int short_limit =cld_is_sptlrpc (cld);int rc;
drivers/staging/lustre/lustre/osc/osc_lock.c +44 -80
148 lines not on this page · raw message
Signature · 1 line
1.7.9.5
PATCH 17/48 staging/lustre/md: fix lu_ucred.c boilerplate
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
In preparing Ie3a3cd99 (LU-1330 obdclass: splits server-side object stack from client) the lu_ucred infrastructure was put in its own file. Fixup the boilerplate of this file to give the proper path, short description, and authors.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1330
Lustre-change: http://review.whamcloud.com/5910
--- drivers/staging/lustre/lustre/obdclass/lu_ucred.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/staging/lustre/lustre/obdclass/lu_ucred.c +4 -4
@@ -33,13 +33,13 @@* This file is part of Lustre, http://www.lustre.org/ * Lustre is a trademark of Sun Microsystems, Inc. * -* lustre/obdclass/lu_object.c +* lustre/obdclass/lu_ucred.c * -* Lustre Object. -* These are the only exported functions, they provide some generic -* infrastructure for managing object devices +* Lustre user credentials context infrastructure. * * Author: Nikita Danilov <[email protected]> +* Author: Fan Yong <[email protected]> +* Author: Vitaly Fertman <[email protected]> */ #define DEBUG_SUBSYSTEM S_CLASS
Signature · 1 line
1.7.9.5
PATCH 18/48 staging/lustre/layout: introduce new layout for released files
In reply to Peng TaoFrom: Jinshan Xiong <[email protected]>
Released files now have a standard layout (with generation, pool, ...) and a stripe count 0 and lmm_pattern flag LOV_PATTERN_F_RELEASED.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2482
Lustre-change: http://review.whamcloud.com/4816
--- .../lustre/lustre/include/lustre/lustre_idl.h | 14 ++-- drivers/staging/lustre/lustre/include/obd.h | 14 ++++ drivers/staging/lustre/lustre/lclient/lcommon_cl.c | 2 +- drivers/staging/lustre/lustre/llite/file.c | 27 +++++--- drivers/staging/lustre/lustre/llite/llite_lib.c | 22 +++--- drivers/staging/lustre/lustre/llite/vvp_io.c | 10 +-- .../staging/lustre/lustre/lov/lov_cl_internal.h | 11 +-- drivers/staging/lustre/lustre/lov/lov_ea.c | 23 ++++-- drivers/staging/lustre/lustre/lov/lov_io.c | 34 +++++++++ drivers/staging/lustre/lustre/lov/lov_obd.c | 2 +- drivers/staging/lustre/lustre/lov/lov_object.c | 73 +++++++++++++++++--- drivers/staging/lustre/lustre/lov/lov_pack.c | 19 +++-- 12 files changed, 197 insertions(+), 54 deletions(-)
drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +10 -4
@@ -1546,10 +1546,16 @@ enum obdo_flags {#define LOV_MAGIC_V1_DEF 0x0CD10BD0#define LOV_MAGIC_V3_DEF 0x0CD30BD0 -#define LOV_PATTERN_RAID0 0x001/* stripes are used round-robin */ -#define LOV_PATTERN_RAID1 0x002/* stripes are mirrors of each other */ -#define LOV_PATTERN_FIRST 0x100/* first stripe is not in round-robin */ -#define LOV_PATTERN_CMOBD 0x200 +#define LOV_PATTERN_RAID0 0x001/* stripes are used round-robin */ +#define LOV_PATTERN_RAID1 0x002/* stripes are mirrors of each other */ +#define LOV_PATTERN_FIRST 0x100/* first stripe is not in round-robin */ +#define LOV_PATTERN_CMOBD 0x200 + +#define LOV_PATTERN_F_MASK 0xffff0000 +#define LOV_PATTERN_F_RELEASED 0x80000000/* HSM released file */ + +#define lov_pattern (pattern) (pattern & ~LOV_PATTERN_F_MASK) +#define lov_pattern_flags (pattern) (pattern & LOV_PATTERN_F_MASK)#define lov_ost_data lov_ost_data_v1struct lov_ost_data_v1 {/* per-stripe data structure (little-endian)*/
drivers/staging/lustre/lustre/include/obd.h +14 -0
@@ -119,6 +119,20 @@ struct lov_stripe_md {#define lsm_stripe_count lsm_wire.lw_stripe_count#define lsm_pool_name lsm_wire.lw_pool_name +static inline bool lsm_is_released (struct lov_stripe_md *lsm) +{ +return !!(lsm->lsm_pattern &LOV_PATTERN_F_RELEASED ); +} + +static inline bool lsm_has_objects (struct lov_stripe_md *lsm) +{ +if (lsm ==NULL ) +return false; +if (lsm_is_released (lsm)) +return false; +return true; +} +struct obd_info ;typedef int (*obd_enqueue_update_f )(void *cookie,int rc);
drivers/staging/lustre/lustre/lclient/lcommon_cl.c +1 -1
@@ -1166,7 +1166,7 @@ int cl_file_inode_init(struct inode *inode, struct lustre_md *md)* locked by I_NEW bit. */ lli->lli_clob = clob; - lli->lli_has_smd = md->lsm !=NULL ; + lli->lli_has_smd =lsm_has_objects (md->lsm);lu_object_ref_add (&clob->co_lu,"inode" , inode); }else result =PTR_ERR (clob);
drivers/staging/lustre/lustre/llite/file.c +16 -11
@@ -1143,7 +1143,7 @@ static int ll_lov_recreate(struct inode *inode, struct ost_id *oi,RETURN (-ENOMEM ); lsm =ccc_inode_lsm_get (inode); -if (lsm ==NULL ) +if (!lsm_has_objects (lsm))GOTO (out, rc = -ENOENT ); lsm_size = sizeof(*lsm) + (sizeof(struct lov_oinfo) *
@@ -1299,6 +1299,12 @@ int ll_lov_getstripe_ea_info(struct inode *inode, const char *filename,* passing it to userspace. */ if (LOV_MAGIC !=cpu_to_le32 (LOV_MAGIC )) { +int stripe_count; + + stripe_count =le16_to_cpu (lmm->lmm_stripe_count); +if (le32_to_cpu (lmm->lmm_pattern) &LOV_PATTERN_F_RELEASED ) + stripe_count =0 ; +/* if function called for directory - we should * avoid swab not existent lsm objects */ if (lmm->lmm_magic ==cpu_to_le32 (LOV_MAGIC_V1 )) {
@@ -1306,13 +1312,13 @@ int ll_lov_getstripe_ea_info(struct inode *inode, const char *filename,if (S_ISREG (body->mode))lustre_swab_lov_user_md_objects ( ((struct lov_user_md_v1 *)lmm)->lmm_objects, - ((struct lov_user_md_v1 *)lmm)->lmm_stripe_count); + stripe_count); }else if (lmm->lmm_magic ==cpu_to_le32 (LOV_MAGIC_V3 )) {lustre_swab_lov_user_md_v3 ((struct lov_user_md_v3 *)lmm);if (S_ISREG (body->mode))lustre_swab_lov_user_md_objects ( ((struct lov_user_md_v3 *)lmm)->lmm_objects, - ((struct lov_user_md_v3 *)lmm)->lmm_stripe_count); + stripe_count); } }
@@ -1698,20 +1704,18 @@ int ll_data_version(struct inode *inode, __u64 *data_version,/* If no stripe, we consider version is 0. */ lsm =ccc_inode_lsm_get (inode); -if (lsm ==NULL ) { +if (!lsm_has_objects (lsm)) { *data_version =0 ;CDEBUG (D_INODE ,"No object for inode\n" ); -RETURN (0 ); +GOTO (out, rc =0 ); }OBD_ALLOC_PTR (obdo); -if (obdo ==NULL ) { -ccc_inode_lsm_put (inode, lsm); -RETURN (-ENOMEM ); - } +if (obdo ==NULL ) +GOTO (out, rc = -ENOMEM ); rc =ll_lsm_getattr (lsm, sbi->ll_dt_exp,NULL , obdo,0 , extent_lock); -if (!rc) { +if (rc ==0 ) {if (!(obdo->o_valid &OBD_MD_FLDATAVERSION )) rc = -EOPNOTSUPP ;else
@@ -1719,8 +1723,9 @@ int ll_data_version(struct inode *inode, __u64 *data_version, }OBD_FREE_PTR (obdo); +EXIT ; +out:ccc_inode_lsm_put (inode, lsm); -RETURN (rc); }
drivers/staging/lustre/lustre/llite/llite_lib.c +13 -9
@@ -1282,13 +1282,6 @@ int ll_md_setattr(struct dentry *dentry, struct md_op_data *op_data,RETURN (rc); } - ia_valid = op_data->op_attr.ia_valid; -/* inode size will be in ll_setattr_ost, can't do it now since dirty -* cache is not cleared yet. */ - op_data->op_attr.ia_valid &= ~(TIMES_SET_FLAGS |ATTR_SIZE ); - rc =simple_setattr (dentry, &op_data->op_attr); - op_data->op_attr.ia_valid = ia_valid; -/* Extract epoch data if obtained. */ op_data->op_handle = md.body->handle; op_data->op_ioepoch = md.body->ioepoch;
@@ -1371,6 +1364,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr)struct ll_inode_info *lli =ll_i2info (inode);struct md_op_data *op_data =NULL ;struct md_open_data *mod =NULL ; +unsigned int valid = attr->ia_valid;int rc =0 , rc1 =0 ;ENTRY ;
@@ -1482,6 +1476,13 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr) rc =ll_setattr_ost (inode, attr);EXIT ; out: +if (rc ==0 ) { +/* Update inode attribute after dirty cache is cleaned +* by truncating OST objects. */ + attr->ia_valid |=ATTR_FORCE ; + rc =simple_setattr (dentry, attr); +LASSERT (rc ==0 ); + }if (op_data) {if (op_data->op_ioepoch) { rc1 =ll_setattr_done_writing (inode, op_data, mod);
@@ -1500,7 +1501,8 @@ out:ll_stats_ops_tally (ll_i2sbi (inode), (attr->ia_valid &ATTR_SIZE ) ?LPROC_LL_TRUNC :LPROC_LL_SETATTR ,1 ); -return rc; + attr->ia_valid = valid; +RETURN (rc); }int ll_setattr (struct dentry *de,struct iattr *attr)
@@ -1923,8 +1925,10 @@ int ll_iocontrol(struct inode *inode, struct file *file, inode->i_flags =ll_ext_to_inode_flags (flags); lsm =ccc_inode_lsm_get (inode); -if (lsm ==NULL ) +if (!lsm_has_objects (lsm)) { +ccc_inode_lsm_put (inode, lsm);RETURN (0 ); + }OBDO_ALLOC (oinfo.oi_oa);if (!oinfo.oi_oa) {
drivers/staging/lustre/lustre/llite/vvp_io.c +6 -4
@@ -407,13 +407,15 @@ static int vvp_io_setattr_start(const struct lu_env *env, {struct cl_io *io = ios->cis_io;struct inode *inode =ccc_object_inode (io->ci_obj); +int result =0 ;mutex_lock (&inode->i_mutex);if (cl_io_is_trunc (io)) -return vvp_io_setattr_trunc (env, ios, inode, - io->u.ci_setattr.sa_attr.lvb_size); -else -return vvp_io_setattr_time (env, ios); + result =vvp_io_setattr_trunc (env, ios, inode, + io->u.ci_setattr.sa_attr.lvb_size); +if (result ==0 ) + result =vvp_io_setattr_time (env, ios); +return result; } static void vvp_io_setattr_end(const struct lu_env *env,
drivers/staging/lustre/lustre/lov/lov_cl_internal.h +7 -4
@@ -162,10 +162,9 @@ struct lov_device {* Layout type. */ enum lov_layout_type { -/** empty file without body */ -LLT_EMPTY , -/** striped file */ -LLT_RAID0 , +LLT_EMPTY ,/** empty file without body (mknod + truncate) */ +LLT_RAID0 ,/** striped file */ +LLT_RELEASED ,/** file with no objects (data in HSM) */ LLT_NR };
@@ -255,6 +254,8 @@ struct lov_object { } raid0;struct lov_layout_state_empty { } empty; +struct lov_layout_state_released { + } released; } u;/** * Thread that acquired lov_object::lo_type_guard in an exclusive
@@ -582,6 +583,8 @@ int lov_io_init_raid0 (const struct lu_env *env, struct cl_object *obj, struct cl_io *io);int lov_io_init_empty (const struct lu_env *env,struct cl_object *obj,struct cl_io *io); +int lov_io_init_released (const struct lu_env *env,struct cl_object *obj, +struct cl_io *io);void lov_lock_unlink (const struct lu_env *env,struct lov_lock_link *link,struct lovsub_lock *sub);
drivers/staging/lustre/lustre/lov/lov_ea.c +19 -4
@@ -57,7 +57,7 @@ struct lovea_unpack_args {static int lsm_lmm_verify_common (struct lov_mds_md *lmm,int lmm_bytes,__u16 stripe_count) { -if (stripe_count ==0 || stripe_count >LOV_V1_INSANE_STRIPE_COUNT ) { +if (stripe_count >LOV_V1_INSANE_STRIPE_COUNT ) {CERROR ("bad stripe count %d\n" , stripe_count);lov_dump_lmm_common (D_WARNING , lmm);return -EINVAL ;
@@ -69,7 +69,7 @@ static int lsm_lmm_verify_common(struct lov_mds_md *lmm, int lmm_bytes,return -EINVAL ; } -if (lmm->lmm_pattern !=cpu_to_le32 (LOV_PATTERN_RAID0 )) { +if (lov_pattern (le32_to_cpu (lmm->lmm_pattern)) !=LOV_PATTERN_RAID0 ) {CERROR ("bad striping pattern\n" );lov_dump_lmm_common (D_WARNING , lmm);return -EINVAL ;
@@ -197,6 +197,8 @@ static int lsm_lmm_verify_v1(struct lov_mds_md_v1 *lmm, int lmm_bytes, } *stripe_count =le16_to_cpu (lmm->lmm_stripe_count); +if (le32_to_cpu (lmm->lmm_pattern) &LOV_PATTERN_F_RELEASED ) + *stripe_count =0 ;if (lmm_bytes <lov_mds_md_size (*stripe_count,LOV_MAGIC_V1 )) { CERROR("LOV EA V1 too small: %d, need %d\n",
@@ -213,11 +215,14 @@ int lsm_unpackmd_v1(struct lov_obd *lov, struct lov_stripe_md *lsm, {struct lov_oinfo *loi;int i; +int stripe_count;__u64 stripe_maxbytes =OBD_OBJECT_EOF ;lsm_unpackmd_common (lsm, lmm); -for (i =0 ; i < lsm->lsm_stripe_count; i++) { + stripe_count =lsm_is_released (lsm) ?0 : lsm->lsm_stripe_count; + +for (i =0 ; i < stripe_count; i++) {/* XXX LOV STACKING call down to osc_unpackmd() */ loi = lsm->lsm_oinfo[i];ostid_le_to_cpu (&lmm->lmm_objects[i].l_ost_oi, &loi->loi_oi);
@@ -240,6 +245,8 @@ int lsm_unpackmd_v1(struct lov_obd *lov, struct lov_stripe_md *lsm, } lsm->lsm_maxbytes = stripe_maxbytes * lsm->lsm_stripe_count; +if (lsm->lsm_stripe_count ==0 ) + lsm->lsm_maxbytes = stripe_maxbytes * lov->desc.ld_tgt_count;return 0 ; }
@@ -267,6 +274,8 @@ static int lsm_lmm_verify_v3(struct lov_mds_md *lmmv1, int lmm_bytes, } *stripe_count =le16_to_cpu (lmm->lmm_stripe_count); +if (le32_to_cpu (lmm->lmm_pattern) &LOV_PATTERN_F_RELEASED ) + *stripe_count =0 ;if (lmm_bytes <lov_mds_md_size (*stripe_count,LOV_MAGIC_V3 )) { CERROR("LOV EA V3 too small: %d, need %d\n",
@@ -285,18 +294,22 @@ int lsm_unpackmd_v3(struct lov_obd *lov, struct lov_stripe_md *lsm,struct lov_mds_md_v3 *lmm;struct lov_oinfo *loi;int i; +int stripe_count;__u64 stripe_maxbytes =OBD_OBJECT_EOF ;int cplen =0 ; lmm = (struct lov_mds_md_v3 *)lmmv1;lsm_unpackmd_common (lsm, (struct lov_mds_md_v1 *)lmm); + + stripe_count =lsm_is_released (lsm) ?0 : lsm->lsm_stripe_count; + cplen =strlcpy (lsm->lsm_pool_name, lmm->lmm_pool_name,sizeof (lsm->lsm_pool_name));if (cplen >=sizeof (lsm->lsm_pool_name))return -E2BIG ; -for (i =0 ; i < lsm->lsm_stripe_count; i++) { +for (i =0 ; i < stripe_count; i++) {/* XXX LOV STACKING call down to osc_unpackmd() */ loi = lsm->lsm_oinfo[i];ostid_le_to_cpu (&lmm->lmm_objects[i].l_ost_oi, &loi->loi_oi);
@@ -319,6 +332,8 @@ int lsm_unpackmd_v3(struct lov_obd *lov, struct lov_stripe_md *lsm, } lsm->lsm_maxbytes = stripe_maxbytes * lsm->lsm_stripe_count; +if (lsm->lsm_stripe_count ==0 ) + lsm->lsm_maxbytes = stripe_maxbytes * lov->desc.ld_tgt_count;return 0 ; }
drivers/staging/lustre/lustre/lov/lov_io.c +34 -0
@@ -964,4 +964,38 @@ int lov_io_init_empty(const struct lu_env *env, struct cl_object *obj,RETURN (result !=0 ); } +int lov_io_init_released (const struct lu_env *env,struct cl_object *obj, +struct cl_io *io) +{ +struct lov_object *lov =cl2lov (obj); +struct lov_io *lio =lov_env_io (env); +int result; +ENTRY ; + +LASSERT (lov->lo_lsm !=NULL ); + lio->lis_object = lov; + +switch (io->ci_type) { +default : +LASSERTF (0 ,"invalid type %d\n" , io->ci_type); +case CIT_MISC : +case CIT_FSYNC : +case CIT_SETATTR : + result =+1 ; +break ; +case CIT_READ : +case CIT_WRITE : +case CIT_FAULT : +/* TODO: need to restore the file. */ + result = -EBADF ; +break ; + } +if (result ==0 ) { +cl_io_slice_add (io, &lio->lis_cl, obj, &lov_empty_io_ops); +atomic_inc (&lov->lo_active_ios); + } + + io->ci_result = result <0 ? result :0 ; +RETURN (result !=0 ); +}/** @} lov */
drivers/staging/lustre/lustre/lov/lov_obd.c +1 -1
@@ -2259,7 +2259,7 @@ static int lov_fiemap(struct lov_obd *lov, __u32 keylen, void *key,int cur_stripe =0 , cur_stripe_wrap =0 , stripe_count;unsigned int buffer_size =FIEMAP_BUFFER_SIZE ; -if (lsm ==NULL ) +if (!lsm_has_objects (lsm))GOTO (out, rc =0 );if (fiemap_count_to_size (fm_key->fiemap.fm_extent_count) < buffer_size)
drivers/staging/lustre/lustre/lov/lov_object.c +63 -10
145 lines not on this page · raw message
drivers/staging/lustre/lustre/lov/lov_pack.c +13 -6
@@ -168,10 +168,12 @@ int lov_packmd(struct obd_export *exp, struct lov_mds_md **lmmp,* to the actual number of OSTs in this filesystem. */ if (!lmmp) { stripe_count =lov_get_stripecnt (lov, lmm_magic, - lsm->lsm_stripe_count); + lsm->lsm_stripe_count); lsm->lsm_stripe_count = stripe_count; - }else { + }else if (!lsm_is_released (lsm)) { stripe_count = lsm->lsm_stripe_count; + }else { + stripe_count =0 ; } }else {/* No need to allocate more than maximum supported stripes.
@@ -325,7 +327,8 @@ int lov_alloc_memmd(struct lov_stripe_md **lsmp, __u16 stripe_count, (*lsmp)->lsm_pattern = pattern; (*lsmp)->lsm_pool_name[0 ] ='\0' ; (*lsmp)->lsm_layout_gen =0 ; - (*lsmp)->lsm_oinfo[0 ]->loi_ost_idx = ~0 ; +if (stripe_count >0 ) + (*lsmp)->lsm_oinfo[0 ]->loi_ost_idx = ~0 ;for (i =0 ; i < stripe_count; i++)loi_init ((*lsmp)->lsm_oinfo[i]);
@@ -359,6 +362,7 @@ int lov_unpackmd(struct obd_export *exp, struct lov_stripe_md **lsmp,int rc =0 , lsm_size;__u16 stripe_count;__u32 magic; +__u32 pattern;ENTRY ;/* If passed an MDS struct use values from there, otherwise defaults */
@@ -384,8 +388,8 @@ int lov_unpackmd(struct obd_export *exp, struct lov_stripe_md **lsmp,RETURN (0 ); } - lsm_size =lov_alloc_memmd (lsmp, stripe_count,LOV_PATTERN_RAID0 , - magic); + pattern =le32_to_cpu (lmm->lmm_pattern); + lsm_size =lov_alloc_memmd (lsmp, stripe_count, pattern, magic);if (lsm_size <0 )RETURN (lsm_size);
@@ -430,7 +434,7 @@ static int __lov_setstripe(struct obd_export *exp, int max_lmm_size, lov->desc.ld_pattern : LOV_PATTERN_RAID0; } -if (lumv1->lmm_pattern !=LOV_PATTERN_RAID0 ) { +if (lov_pattern (lumv1->lmm_pattern) !=LOV_PATTERN_RAID0 ) {CDEBUG (D_IOCTL ,"bad userland stripe pattern: %#x\n" , lumv1->lmm_pattern);RETURN (-EINVAL );
@@ -490,6 +494,9 @@ static int __lov_setstripe(struct obd_export *exp, int max_lmm_size, } } +if (lumv1->lmm_pattern &LOV_PATTERN_F_RELEASED ) + stripe_count =0 ; + rc =lov_alloc_memmd (lsmp, stripe_count, lumv1->lmm_pattern, lmm_magic);if (rc >=0 ) {
Signature · 1 line
1.7.9.5
PATCH 19/48 staging/lustre/mdt: add macros for fid string len
In reply to Peng TaoFrom: jcl <[email protected]>
add 2 macros for the length of a fid string 0xSEQ:0xOID:0xVER and it's brace version (FID_NOBRACE_LEN, and FID_LEN)
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2782
Lustre-change: http://review.whamcloud.com/5299
--- .../lustre/lustre/include/lustre/lustre_user.h | 2 ++ 1 file changed, 2 insertions(+)
drivers/staging/lustre/lustre/include/lustre/lustre_user.h +2 -0
@@ -462,6 +462,8 @@ static inline void obd_uuid2fsname(char *buf, char *uuid, int buflen)/* printf display format e.g. printf("file FID is "DFID"\n", PFID(fid)); */ +#define FID_NOBRACE_LEN 40 +#define FID_LEN (FID_NOBRACE_LEN + 2)#define DFID_NOBRACE LPX64":0x%x:0x%x"#define DFID "["DFID_NOBRACE"]"#define PFID (fid) \
Signature · 1 line
1.7.9.5
PATCH 20/48 staging/lustre/llapi: add user space method for lov_user_md
In reply to Peng TaoFrom: jcl <[email protected]>
move lov_mds_md_size from obd_lov.h to lustre_idl.h to have it close to lov_mds_md definition.
add lov_user_md_size() to compute lum size so llapi and user space utils do not use kernel internal definitions/methods
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3345
Lustre-change: http://review.whamcloud.com/6345
--- .../lustre/lustre/include/lustre/lustre_idl.h | 11 +++++++++++ .../lustre/lustre/include/lustre/lustre_user.h | 10 ++++++++++ drivers/staging/lustre/lustre/include/obd_lov.h | 10 ---------- 3 files changed, 21 insertions(+), 10 deletions(-)
drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +11 -0
@@ -1670,6 +1670,17 @@ struct lov_mds_md_v3 { /* LOV EA mds/wire data (little-endian) */struct lov_ost_data_v1 lmm_objects[0 ];/* per-stripe data */ }; +static inline __u32 lov_mds_md_size (__u16 stripes,__u32 lmm_magic) +{ +if (lmm_magic ==LOV_MAGIC_V3 ) +return sizeof (struct lov_mds_md_v3 ) + + stripes *sizeof (struct lov_ost_data_v1 ); +else +return sizeof (struct lov_mds_md_v1 ) + + stripes *sizeof (struct lov_ost_data_v1 ); +} + +#define OBD_MD_FLID (0x00000001ULL)/* object ID */ #define OBD_MD_FLATIME (0x00000002ULL)/* access time */ #define OBD_MD_FLMTIME (0x00000004ULL)/* data modification time */
drivers/staging/lustre/lustre/include/lustre/lustre_user.h +10 -0
@@ -347,6 +347,16 @@ struct lov_user_md_v3 { /* LOV EA user data (host-endian) */struct lov_user_ost_data_v1 lmm_objects[0 ];/* per-stripe data */ } __attribute__((packed)); +static inline __u32 lov_user_md_size (__u16 stripes,__u32 lmm_magic) +{ +if (lmm_magic ==LOV_USER_MAGIC_V3 ) +return sizeof (struct lov_user_md_v3 ) + + stripes *sizeof (struct lov_user_ost_data_v1 ); +else +return sizeof (struct lov_user_md_v1 ) + + stripes *sizeof (struct lov_user_ost_data_v1 ); +} +/* Compile with -D_LARGEFILE64_SOURCE or -D_GNU_SOURCE (or #define) to * use this. It is unsafe to #define those values in this header as it * is possible the application has already #included <sys/stat.h>. */
drivers/staging/lustre/lustre/include/obd_lov.h +0 -10
@@ -44,16 +44,6 @@ static inline int lov_stripe_md_size(__u16 stripes)return sizeof (struct lov_stripe_md ) + stripes*sizeof (struct lov_oinfo *); } -static inline __u32 lov_mds_md_size (__u16 stripes,__u32 lmm_magic) -{ -if (lmm_magic ==LOV_MAGIC_V3 ) -return sizeof (struct lov_mds_md_v3 ) + - stripes *sizeof (struct lov_ost_data_v1 ); -else -return sizeof (struct lov_mds_md_v1 ) + - stripes *sizeof (struct lov_ost_data_v1 ); -} -struct lov_version_size {__u32 lvs_magic;size_t lvs_lmm_size;
Signature · 1 line
1.7.9.5
PATCH 21/48 staging/lustre/obdclass: use a dummy structure for lu_ref_link
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
Move the definition of struct lu_ref_link to lu_ref.h. If USE_LU_REF is not defined then define it to be the empty struct. Change the return type of lu_ref_add() and lu_ref_add_atomic() to void. Add lu_ref_add_at() taking same arguments as lu_ref_add() togerther with a pointer to a struct lu_ref_link and returning void. Adjust all structures containing a lu_ref_link pointer to contain a struct lu_ref_link instead. Use lu_ref_add_at() and lu_ref_del_at() to handle embedded lu_ref_links.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3059
Lustre-change: http://review.whamcloud.com/5920
--- drivers/staging/lustre/lustre/include/cl_object.h | 10 +-- drivers/staging/lustre/lustre/include/lu_object.h | 18 +++-- drivers/staging/lustre/lustre/include/lu_ref.h | 77 ++++++++++++-------- drivers/staging/lustre/lustre/obdclass/cl_io.c | 17 +++-- drivers/staging/lustre/lustre/obdclass/cl_lock.c | 6 +- drivers/staging/lustre/lustre/obdclass/cl_page.c | 5 +- drivers/staging/lustre/lustre/obdclass/lu_object.c | 15 ++-- drivers/staging/lustre/lustre/obdclass/lu_ref.c | 74 +++++++++---------- drivers/staging/lustre/lustre/osc/osc_cache.c | 8 +- 9 files changed, 130 insertions(+), 100 deletions(-)
drivers/staging/lustre/lustre/include/cl_object.h +5 -5
@@ -768,11 +768,11 @@ struct cl_page {/** List of references to this page, for debugging. */ struct lu_ref cp_reference;/** Link to an object, for debugging. */ -struct lu_ref_link *cp_obj_ref; +struct lu_ref_link cp_obj_ref;/** Link to a queue, for debugging. */ -struct lu_ref_link *cp_queue_ref; +struct lu_ref_link cp_queue_ref;/** Per-page flags from enum cl_page_flags. Protected by a VM lock. */ -unsigned cp_flags; +unsigned cp_flags;/** Assigned if doing a sync_io */ struct cl_sync_io *cp_sync_io; };
@@ -1625,7 +1625,7 @@ struct cl_lock {/** * A reference for cl_lock::cll_descr::cld_obj. For debugging. */ -struct lu_ref_link *cll_obj_ref; +struct lu_ref_link cll_obj_ref;#ifdef CONFIG_LOCKDEP /* "dep_map" name is assumed by lockdep.h macros. */ struct lockdep_map dep_map;
@@ -2517,7 +2517,7 @@ struct cl_req_obj {/** object itself */ struct cl_object *ro_obj;/** reference to cl_req_obj::ro_obj. For debugging. */ -struct lu_ref_link *ro_obj_ref; +struct lu_ref_link ro_obj_ref;/* something else? Number of pages for a given object? */ };
drivers/staging/lustre/lustre/include/lu_object.h +13 -5
@@ -496,7 +496,7 @@ struct lu_object {/** * Link to the device, for debugging. */ -struct lu_ref_link *lo_dev_ref; +struct lu_ref_link lo_dev_ref; };enum lu_object_header_flags {
@@ -868,11 +868,19 @@ static inline __u32 lu_object_attr(const struct lu_object *o)return o->lo_header->loh_attr; } -static inline struct lu_ref_link *lu_object_ref_add (struct lu_object *o, -const char *scope, -const void *source) +static inline void lu_object_ref_add (struct lu_object *o, +const char *scope, +const void *source) { -return lu_ref_add (&o->lo_header->loh_reference, scope, source); +lu_ref_add (&o->lo_header->loh_reference, scope, source); +} + +static inline void lu_object_ref_add_at (struct lu_object *o, +struct lu_ref_link *link, +const char *scope, +const void *source) +{ +lu_ref_add_at (&o->lo_header->loh_reference, link, scope, source); } static inline void lu_object_ref_del(struct lu_object *o,
drivers/staging/lustre/lustre/include/lu_ref.h +48 -29
106 lines not on this page · raw message
drivers/staging/lustre/lustre/obdclass/cl_io.c +9 -8
@@ -1105,7 +1105,7 @@ void cl_page_list_add(struct cl_page_list *plist, struct cl_page *page)LASSERT (list_empty (&page->cp_batch));list_add_tail (&page->cp_batch, &plist->pl_pages); ++plist->pl_nr; - page->cp_queue_ref =lu_ref_add (&page->cp_reference,"queue" , plist); +lu_ref_add_at (&page->cp_reference, &page->cp_queue_ref,"queue" , plist);cl_page_get (page);EXIT ; }
@@ -1126,7 +1126,7 @@ void cl_page_list_del(const struct lu_env *env,mutex_unlock (&page->cp_mutex);lockdep_on (); --plist->pl_nr; -lu_ref_del_at (&page->cp_reference, page->cp_queue_ref,"queue" , plist); +lu_ref_del_at (&page->cp_reference, &page->cp_queue_ref,"queue" , plist);cl_page_put (env, page);EXIT ; }
@@ -1146,8 +1146,8 @@ void cl_page_list_move(struct cl_page_list *dst, struct cl_page_list *src,list_move_tail (&page->cp_batch, &dst->pl_pages); --src->pl_nr; ++dst->pl_nr; -lu_ref_set_at (&page->cp_reference, - page->cp_queue_ref,"queue" , src, dst); +lu_ref_set_at (&page->cp_reference, &page->cp_queue_ref,"queue" , + src, dst);EXIT ; }EXPORT_SYMBOL (cl_page_list_move);
@@ -1202,7 +1202,8 @@ void cl_page_list_disown(const struct lu_env *env,* XXX cl_page_disown0() will fail if page is not locked. */ cl_page_disown0 (env, io, page); -lu_ref_del (&page->cp_reference,"queue" , plist); +lu_ref_del_at (&page->cp_reference, &page->cp_queue_ref,"queue" , + plist);cl_page_put (env, page); }EXIT ;
@@ -1449,7 +1450,7 @@ static void cl_req_free(const struct lu_env *env, struct cl_req *req)struct cl_object *obj = req->crq_o[i].ro_obj;if (obj !=NULL ) {lu_object_ref_del_at (&obj->co_lu, - req->crq_o[i].ro_obj_ref, + &req->crq_o[i].ro_obj_ref,"cl_req" , req);cl_object_put (env, obj); }
@@ -1570,8 +1571,8 @@ void cl_req_page_add(const struct lu_env *env,if (rqo->ro_obj ==NULL ) { rqo->ro_obj = obj;cl_object_get (obj); - rqo->ro_obj_ref =lu_object_ref_add (&obj->co_lu, -"cl_req" , req); +lu_object_ref_add_at (&obj->co_lu, &rqo->ro_obj_ref, +"cl_req" , req);break ; } }
drivers/staging/lustre/lustre/obdclass/cl_lock.c +3 -3
@@ -267,7 +267,7 @@ static void cl_lock_free(const struct lu_env *env, struct cl_lock *lock) }CS_LOCK_DEC (obj, total);CS_LOCKSTATE_DEC (obj, lock->cll_state); -lu_object_ref_del_at (&obj->co_lu, lock->cll_obj_ref,"cl_lock" , lock); +lu_object_ref_del_at (&obj->co_lu, &lock->cll_obj_ref,"cl_lock" , lock);cl_object_put (env, obj);lu_ref_fini (&lock->cll_reference);lu_ref_fini (&lock->cll_holders);
@@ -373,8 +373,8 @@ static struct cl_lock *cl_lock_alloc(const struct lu_env *env, lock->cll_descr = *descr; lock->cll_state =CLS_NEW ;cl_object_get (obj); - lock->cll_obj_ref =lu_object_ref_add (&obj->co_lu, -"cl_lock" , lock); +lu_object_ref_add_at (&obj->co_lu, &lock->cll_obj_ref,"cl_lock" , + lock);INIT_LIST_HEAD (&lock->cll_layers);INIT_LIST_HEAD (&lock->cll_linkage);INIT_LIST_HEAD (&lock->cll_inclosure);
drivers/staging/lustre/lustre/obdclass/cl_page.c +3 -2
@@ -270,7 +270,7 @@ static void cl_page_free(const struct lu_env *env, struct cl_page *page) }CS_PAGE_DEC (obj, total);CS_PAGESTATE_DEC (obj, page->cp_state); -lu_object_ref_del_at (&obj->co_lu, page->cp_obj_ref,"cl_page" , page); +lu_object_ref_del_at (&obj->co_lu, &page->cp_obj_ref,"cl_page" , page);cl_object_put (env, obj);lu_ref_fini (&page->cp_reference);OBD_FREE (page, pagesize);
@@ -305,7 +305,8 @@ static struct cl_page *cl_page_alloc(const struct lu_env *env,atomic_inc (&page->cp_ref); page->cp_obj = o;cl_object_get (o); - page->cp_obj_ref =lu_object_ref_add (&o->co_lu,"cl_page" ,page); +lu_object_ref_add_at (&o->co_lu, &page->cp_obj_ref,"cl_page" , + page); page->cp_index = ind;cl_page_state_set_trust (page,CPS_CACHED ); page->cp_type = type;
drivers/staging/lustre/lustre/obdclass/lu_object.c +8 -7
@@ -1147,15 +1147,16 @@ EXPORT_SYMBOL(lu_device_fini);* Initialize object \a o that is part of compound object \a h and was created * by device \a d. */ -int lu_object_init (struct lu_object *o, -struct lu_object_header *h,struct lu_device *d) +int lu_object_init (struct lu_object *o,struct lu_object_header *h, +struct lu_device *d) { -memset (o,0 ,sizeof *o); +memset (o,0 ,sizeof (*o)); o->lo_header = h; - o->lo_dev = d; + o->lo_dev = d;lu_device_get (d); - o->lo_dev_ref =lu_ref_add (&d->ld_reference,"lu_object" , o); +lu_ref_add_at (&d->ld_reference, &o->lo_dev_ref,"lu_object" , o);INIT_LIST_HEAD (&o->lo_linkage); +return 0 ; }EXPORT_SYMBOL (lu_object_init);
@@ -1170,8 +1171,8 @@ void lu_object_fini(struct lu_object *o)LASSERT (list_empty (&o->lo_linkage));if (dev !=NULL ) { -lu_ref_del_at (&dev->ld_reference, - o->lo_dev_ref ,"lu_object" , o); +lu_ref_del_at (&dev->ld_reference, &o->lo_dev_ref, +"lu_object" , o);lu_device_put (dev); o->lo_dev =NULL ; }
drivers/staging/lustre/lustre/obdclass/lu_ref.c +36 -38
126 lines not on this page · raw message
drivers/staging/lustre/lustre/osc/osc_cache.c +5 -3
@@ -2064,7 +2064,7 @@ static void osc_check_rpcs(const struct lu_env *env, struct client_obd *cli,while ((osc =osc_next_obj (cli)) !=NULL ) {struct cl_object *obj =osc2cl (osc); -struct lu_ref_link *link; +struct lu_ref_link link;OSC_IO_DEBUG (osc,"%lu in flight\n" ,rpcs_in_flight (cli));
@@ -2075,7 +2075,8 @@ static void osc_check_rpcs(const struct lu_env *env, struct client_obd *cli,cl_object_get (obj);client_obd_list_unlock (&cli->cl_loi_list_lock); - link =lu_object_ref_add (&obj->co_lu,"check" , current); +lu_object_ref_add_at (&obj->co_lu, &link,"check" , + current);/* attempt some read/write balancing by alternating between * reads and writes in an object. The makes_rpc checks here
@@ -2116,7 +2117,8 @@ static void osc_check_rpcs(const struct lu_env *env, struct client_obd *cli,osc_object_unlock (osc);osc_list_maint (cli, osc); -lu_object_ref_del_at (&obj->co_lu, link,"check" , current); +lu_object_ref_del_at (&obj->co_lu, &link,"check" , + current);cl_object_put (env, obj);client_obd_list_lock (&cli->cl_loi_list_lock);
Signature · 1 line
1.7.9.5
Greg Kroah-Hartman
maintainerRe: [PATCH 21/48] staging/lustre/obdclass: use a dummy structure for lu_ref_link
Quoted 13 lines · Peng Tao
From: "John L. Hammond" <[email protected]>
Move the definition of struct
lu_ref_linktolu_ref.h. IfUSE_LU_REFis not defined then define it to be the empty struct. Change the return type oflu_ref_add()andlu_ref_add_atomic()to void. Addlu_ref_add_at()taking same arguments aslu_ref_add()togerther with a pointer to a structlu_ref_linkand returning void. Adjust all structures containing alu_ref_linkpointer to contain a structlu_ref_linkinstead. Uselu_ref_add_at()andlu_ref_del_at()to handle embeddedlu_ref_links.Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3059
Lustre-change: http://review.whamcloud.com/5920
Quoted 7 lines
Signed-off-by John L. Hammond <[email protected]>Reviewed-by Alex Zhuravlev <[email protected]>Reviewed-by Jinshan Xiong <[email protected]>Reviewed-by Mike Pershin <[email protected]>Reviewed-by Oleg Drokin <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Andreas Dilger <[email protected]>
This patch fails to apply :(
Re: [PATCH 21/48] staging/lustre/obdclass: use a dummy structure for lu_ref_link
Quoted 22 lines · Greg Kroah-Hartman
Quoted 13 lines · Peng Tao
From: "John L. Hammond" <[email protected]>
Move the definition of struct
lu_ref_linktolu_ref.h. IfUSE_LU_REFis not defined then define it to be the empty struct. Change the return type oflu_ref_add()andlu_ref_add_atomic()to void. Addlu_ref_add_at()taking same arguments aslu_ref_add()togerther with a pointer to a structlu_ref_linkand returning void. Adjust all structures containing alu_ref_linkpointer to contain a structlu_ref_linkinstead. Uselu_ref_add_at()andlu_ref_del_at()to handle embeddedlu_ref_links.Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3059
Lustre-change: http://review.whamcloud.com/5920Quoted 7 lines
Signed-off-by John L. Hammond <[email protected]>Reviewed-by Alex Zhuravlev <[email protected]>Reviewed-by Jinshan Xiong <[email protected]>Reviewed-by Mike Pershin <[email protected]>Reviewed-by Oleg Drokin <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Andreas Dilger <[email protected]>
Quoted 2 lines
This patch fails to apply :(
It relies on one previous patch ([PATCH 02/48] staging/lustre/obdclass: restore lu_ref), which you have suggested to drop. I will remove unnecessary part of the patch and resend (if there's still anything left).
Thanks,
Tao
PATCH 22/48 staging/lustre: fix 'code maintainability' errors
In reply to Peng TaoFrom: Sebastien Buisson <[email protected]>
Fix 'code maintainability' issues found by Coverity version 6.5.1:
Unused pointer value (UNUSED_VALUE)
Pointer returned by function is never used.
Missing varargs init or cleanup (VARARGS)va_end was not called for variable.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3107
Lustre-change: http://review.whamcloud.com/5944
--- .../lustre/lustre/obdclass/lprocfs_jobstats.c | 2 -- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 1 + 2 files changed, 1 insertion(+), 2 deletions(-)
drivers/staging/lustre/lustre/obdclass/lprocfs_jobstats.c +0 -2
@@ -366,7 +366,6 @@ static int lprocfs_jobstats_seq_show(struct seq_file *p, void *v)struct job_stat *job = v;struct lprocfs_stats *s;struct lprocfs_counter ret; -struct lprocfs_counter *cntr;struct lprocfs_counter_header *cntr_header;int i;
@@ -380,7 +379,6 @@ static int lprocfs_jobstats_seq_show(struct seq_file *p, void *v) s = job->js_stats;for (i =0 ; i < s->ls_num; i++) { - cntr =lprocfs_stats_counter_get (s,0 , i); cntr_header = &s->ls_cnt_header[i];lprocfs_stats_collect (s, i, &ret);
drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +1 -0
@@ -2462,6 +2462,7 @@ void _debug_req(struct ptlrpc_request *req, rep_ok ?lustre_msg_get_flags (req->rq_repmsg) :-1 , req->rq_status, rep_ok ? lustre_msg_get_status(req->rq_repmsg) : -1); +va_end (args); }EXPORT_SYMBOL (_debug_req);
Signature · 1 line
1.7.9.5
PATCH 23/48 staging/lustre/fid: prepare FID module for client server split
In reply to Peng TaoFrom: Liu Xuezhao <[email protected]>
Split FID server from client, fid_{handler,store,lib}.c are not compliled unless server support is enabled. Generally cleanup includes in lustre/fid/ and reduce the need for client code to directly or indirectly include {dt,md}_object.h.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1330
Lustre-change: http://review.whamcloud.com/2673
--- drivers/staging/lustre/lustre/fid/Makefile | 2 +- drivers/staging/lustre/lustre/fid/fid_handler.c | 661 -------------------- drivers/staging/lustre/lustre/fid/fid_internal.h | 36 +- drivers/staging/lustre/lustre/fid/fid_lib.c | 14 +- drivers/staging/lustre/lustre/fid/fid_request.c | 133 ++-- drivers/staging/lustre/lustre/fid/fid_store.c | 259 -------- drivers/staging/lustre/lustre/fid/lproc_fid.c | 3 - drivers/staging/lustre/lustre/include/lu_object.h | 5 + drivers/staging/lustre/lustre/include/lustre_fid.h | 24 +- drivers/staging/lustre/lustre/include/md_object.h | 5 - drivers/staging/lustre/lustre/include/obd.h | 7 +- 11 files changed, 131 insertions(+), 1018 deletions(-) delete mode 100644 drivers/staging/lustre/lustre/fid/fid_handler.c delete mode 100644 drivers/staging/lustre/lustre/fid/fid_store.c
drivers/staging/lustre/lustre/fid/Makefile +1 -1
@@ -1,5 +1,5 @@ obj-$(CONFIG_LUSTRE_FS) += fid.o -fid-y := fid_handler.o fid_store.o fid_request.o lproc_fid.o fid_lib.o +fid-y := fid_request.o lproc_fid.o fid_lib.o ccflags-y := -I$(src)/../include
drivers/staging/lustre/lustre/fid/fid_handler.c deleted +0 -661
661 lines not on this page · raw message
drivers/staging/lustre/lustre/fid/fid_internal.h +4 -32
@@ -41,44 +41,16 @@#define __FID_INTERNAL_H#include <lustre/lustre_idl.h> -#include <dt_object.h> -#include <linux/libcfs/libcfs.h> -struct seq_thread_info { -struct req_capsule *sti_pill; -struct lu_seq_range sti_space; -struct lu_buf sti_buf; -}; - -enum { -SEQ_TXN_STORE_CREDITS =20 -}; - -extern struct lu_context_key seq_thread_key; - +/* Functions used internally in module. */ int seq_client_alloc_super (struct lu_client_seq *seq,const struct lu_env *env); -/* Store API functions. */ -int seq_store_init (struct lu_server_seq *seq, -const struct lu_env *env, -struct dt_device *dt); -void seq_store_fini (struct lu_server_seq *seq, -const struct lu_env *env); - -int seq_store_read (struct lu_server_seq *seq, -const struct lu_env *env); - -int seq_store_update (const struct lu_env *env,struct lu_server_seq *seq, -struct lu_seq_range *out,int sync); - -#ifdef LPROCFS -extern struct lprocfs_vars seq_server_proc_list[]; +# ifdef LPROCFS extern struct lprocfs_vars seq_client_proc_list[]; -#endif - +# endif -extern proc_dir_entry_t *seq_type_proc_dir; +extern struct proc_dir_entry *seq_type_proc_dir;#endif /* __FID_INTERNAL_H */
drivers/staging/lustre/lustre/fid/fid_lib.c +6 -8
@@ -43,11 +43,9 @@#define DEBUG_SUBSYSTEM S_FID -# include <linux/libcfs/libcfs.h> -# include <linux/module.h> - -#include <obd.h> -#include <lu_object.h> +#include <linux/libcfs/libcfs.h> +#include <linux/module.h> +#include <lustre/lustre_idl.h> #include <lustre_fid.h> /**
@@ -56,9 +54,9 @@* * Fid namespace: * <pre> -* Normal FID: seq:64 [2^33,2^64-1] oid:32 ver:32 -* IGIF : 0:32, ino:32 gen:32 0:32 -* IDIF : 0:31, 1:1, ost-index:16, objd:48 0:32 +* Normal FID: seq:64 [2^33,2^64-1] oid:32 ver:32 +* IGIF : 0:32, ino:32 gen:32 0:32 +* IDIF : 0:31, 1:1, ost-index:16, objd:48 0:32 * </pre> * * The first 0x400 sequences of normal FID are reserved for special purpose.
drivers/staging/lustre/lustre/fid/fid_request.c +97 -36
194 lines not on this page · raw message
drivers/staging/lustre/lustre/fid/fid_store.c deleted +0 -259
259 lines not on this page · raw message
drivers/staging/lustre/lustre/fid/lproc_fid.c +0 -3
@@ -204,9 +204,6 @@ lprocfs_fid_server_seq_show(struct seq_file *m, void *unused)RETURN (rc); } -struct lprocfs_vars seq_server_proc_list[] = { -}; -LPROC_SEQ_FOPS (lprocfs_fid_space);LPROC_SEQ_FOPS (lprocfs_fid_width);LPROC_SEQ_FOPS_RO (lprocfs_fid_server);
drivers/staging/lustre/lustre/include/lu_object.h +5 -0
@@ -665,6 +665,11 @@ lu_site_bkt_from_fid(struct lu_site *site, struct lu_fid *fid)return cfs_hash_bd_extra_get (site->ls_obj_hash, &bd); } +static inline struct seq_server_site *lu_site2seq (const struct lu_site *s) +{ +return s->ld_seq_site; +} +/** \name ctors * Constructors/destructors. * @{
drivers/staging/lustre/lustre/include/lustre_fid.h +17 -7
@@ -38,8 +38,8 @@* Author: Yury Umanets <[email protected]> */ -#ifndef __LINUX_FID_H -#define __LINUX_FID_H +#ifndef __LUSTRE_FID_H +#define __LUSTRE_FID_H/** \defgroup fid fid *
@@ -154,13 +154,12 @@#include <linux/libcfs/libcfs.h> #include <lustre/lustre_idl.h> -#include <lustre_req_layout.h> -#include <lustre_mdt.h> -#include <obd.h> - +struct lu_env ;struct lu_site ;struct lu_context ; +struct obd_device ; +struct obd_export ;/* Whole sequences space range and zero range definitions */ extern const struct lu_seq_range LUSTRE_SEQ_SPACE_RANGE ;
@@ -320,6 +319,12 @@ static inline void lu_last_id_fid(struct lu_fid *fid, __u64 seq) fid->f_ver =0 ; } +/* seq client type */ +enum lu_cli_type { +LUSTRE_SEQ_METADATA =1 , +LUSTRE_SEQ_DATA +}; +enum lu_mgr_type {LUSTRE_SEQ_SERVER ,LUSTRE_SEQ_CONTROLLER
@@ -426,10 +431,14 @@ struct lu_server_seq {struct seq_server_site *lss_site; }; +struct com_thread_info ;int seq_query (struct com_thread_info *info); + +struct ptlrpc_request ;int seq_handle (struct ptlrpc_request *req);/* Server methods */ + int seq_server_init(struct lu_server_seq *seq, struct dt_device *dev, const char *prefix,
@@ -472,6 +481,7 @@ int seq_site_fini(const struct lu_env *env, struct seq_server_site *ss);int fid_is_local (const struct lu_env *env,struct lu_site *site,const struct lu_fid *fid); +enum lu_cli_type ;int client_fid_init (struct obd_device *obd,struct obd_export *exp,enum lu_cli_type type);int client_fid_fini (struct obd_device *obd);
@@ -760,4 +770,4 @@ static inline void range_be_to_cpu(struct lu_seq_range *dst, const struct lu_seq/** @} fid */ -#endif /* __LINUX_FID_H */ +#endif /* __LUSTRE_FID_H */
drivers/staging/lustre/lustre/include/md_object.h +0 -5
@@ -503,11 +503,6 @@ static inline struct md_device *md_obj2dev(const struct md_object *o) return container_of0(o->mo_lu.lo_dev, struct md_device, md_lu_dev); } -static inline struct seq_server_site *lu_site2seq (const struct lu_site *s) -{ -return s->ld_seq_site; -} -static inline int md_device_init (struct md_device *md,struct lu_device_type *t) {return lu_device_init (&md->md_lu_dev, t);
drivers/staging/lustre/lustre/include/obd.h +1 -6
@@ -52,6 +52,7 @@#include <lu_ref.h> #include <lustre_lib.h> #include <lustre_export.h> +#include <lustre_fid.h> #include <lustre_fld.h> #include <lustre_capa.h>
@@ -1232,12 +1233,6 @@ typedef int (* md_enqueue_cb_t)(struct ptlrpc_request *req,struct md_enqueue_info *minfo, int rc); -/* seq client type */ -enum lu_cli_type { -LUSTRE_SEQ_METADATA =1 , -LUSTRE_SEQ_DATA -}; -struct md_enqueue_info {struct md_op_data mi_data;struct lookup_intent mi_it;
Signature · 1 line
1.7.9.5
PATCH 24/48 staging/lustre/llite: force lvb_data update after layout change
In reply to Peng TaoFrom: jcl <[email protected]>
When a file is restored the layout lock is first associated with the released layout and after restore it has to be assocaited with the new layout. This patch forces lvb_data update in ll_layout_fetch() even if one is present (case for released->normal state change)
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3299
Lustre-change: http://review.whamcloud.com/6291
--- drivers/staging/lustre/lustre/llite/file.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-)
drivers/staging/lustre/lustre/llite/file.c +11 -9
@@ -2941,7 +2941,11 @@ static int ll_layout_fetch(struct inode *inode, struct ldlm_lock *lock)int rc;ENTRY ; -if (lock->l_lvb_data !=NULL ) +CDEBUG (D_INODE ,DFID " LVB_READY=%d l_lvb_data=%p l_lvb_len=%d\n" , +PFID (ll_inode2fid (inode)), !!(lock->l_flags &LDLM_FL_LVB_READY ), + lock->l_lvb_data, lock->l_lvb_len); + +if ((lock->l_lvb_data !=NULL ) && (lock->l_flags &LDLM_FL_LVB_READY ))RETURN (0 );/* if layout lock was granted right away, the layout is returned
@@ -2977,15 +2981,13 @@ static int ll_layout_fetch(struct inode *inode, struct ldlm_lock *lock)memcpy (lvbdata, lmm, lmmsize);lock_res_and_lock (lock); -if (lock->l_lvb_data ==NULL ) { - lock->l_lvb_data = lvbdata; - lock->l_lvb_len = lmmsize; - lvbdata =NULL ; - } +if (lock->l_lvb_data !=NULL ) +OBD_FREE_LARGE (lock->l_lvb_data, lock->l_lvb_len); + + lock->l_lvb_data = lvbdata; + lock->l_lvb_len = lmmsize;unlock_res_and_lock (lock); -if (lvbdata !=NULL ) -OBD_FREE_LARGE (lvbdata, lmmsize);EXIT ; out:
@@ -3017,7 +3019,7 @@ static int ll_layout_lock_set(struct lustre_handle *lockh, ldlm_mode_t mode,LASSERT (ldlm_has_layout (lock));LDLM_DEBUG (lock,"File %p/" DFID " being reconfigured: %d.\n" , - inode,PFID (&lli->lli_fid), reconf); + inode,PFID (&lli->lli_fid), reconf);/* in case this is a caching lock and reinstate with new inode */ md_set_lock_data (sbi->ll_md_exp, &lockh->cookie, inode,NULL );
Signature · 1 line
1.7.9.5
PATCH 25/48 staging/lustre/lfsck: LFSCK 1.5 technical debts (3)
In reply to Peng TaoFrom: Fan Yong <[email protected]>
Original patch resolves some LFSCK 1.5 technical debts, including:
1) Check and remove repeated linkea entries.
2) Merge some "goto" branches to make the code more readable.
3) Some comments about object's nlink inconsistency processing.
This patch picks the obd flags change.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2915
Lustre-change: http://review.whamcloud.com/6344
--- .../staging/lustre/lustre/include/obd_support.h | 1 + 1 file changed, 1 insertion(+)
drivers/staging/lustre/lustre/include/obd_support.h +1 -0
@@ -470,6 +470,7 @@ int obd_alloc_fail(const void *ptr, const char *name, const char *type,#define OBD_FAIL_LFSCK_DELAY3 0x1602#define OBD_FAIL_LFSCK_LINKEA_CRASH 0x1603#define OBD_FAIL_LFSCK_LINKEA_MORE 0x1604 +#define OBD_FAIL_LFSCK_LINKEA_MORE2 0x1605#define OBD_FAIL_LFSCK_FATAL1 0x1608#define OBD_FAIL_LFSCK_FATAL2 0x1609#define OBD_FAIL_LFSCK_CRASH 0x160a
Signature · 1 line
1.7.9.5
PATCH 26/48 staging/lustre/osc: Check return code for lu_kmem_init
In reply to Peng TaoFrom: Keith Mannthey <[email protected]>
lu_kmem_init can fail and returns has a return code. Check for this return code in lu_kmem_init.
This issue was found during 2gb VM Racer testing
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3063
Lustre-change: http://review.whamcloud.com/6514
--- drivers/staging/lustre/lustre/osc/osc_request.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/staging/lustre/lustre/osc/osc_request.c +2 -0
@@ -3681,6 +3681,8 @@ int __init osc_init(void)CDEBUG (D_INFO ,"Lustre OSC module (%p).\n" , &osc_caches); rc =lu_kmem_init (osc_caches); +if (rc) +RETURN (rc);lprocfs_osc_init_vars (&lvars);
Signature · 1 line
1.7.9.5
PATCH 27/48 staging/lustre/ptlrpc: Race between start and stop service threads
In reply to Peng TaoFrom: Hiroya Nozaki <[email protected]>
When ptlrpc_start_thread fails to create a new thread, it will finalize and free a struct ptlrpc_thread created and used here. Considering this, it can be a problem when ptlrpc_svcpt_stop_thread is driven and handles the struct ptlrpc_thread right after or right before failure of cfs_create_thread. Because this situation let the both of ptlrpc_start_thread and ptlrpc_svcpt_stop_threads access the freed ptlrpc_thread and cause OS panic. Or, it may happen that ptlrpc_svcpt_stop_threads waits forever holding an already-freed waitq.
This patch adds an error handling into ptlrpc_start_thread to fix this problem.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2889
Lustre-change: http://review.whamcloud.com/5552
--- drivers/staging/lustre/lustre/ptlrpc/service.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
drivers/staging/lustre/lustre/ptlrpc/service.c +12 -4
@@ -2765,11 +2765,19 @@ int ptlrpc_start_thread(struct ptlrpc_service_part *svcpt, int wait)CERROR ("cannot start thread '%s': rc %d\n" , thread->t_name, rc);spin_lock (&svcpt->scp_lock); -list_del (&thread->t_link); --svcpt->scp_nthrs_starting; -spin_unlock (&svcpt->scp_lock); - -OBD_FREE (thread,sizeof (*thread)); +if (thread_is_stopping (thread)) { +/* this ptlrpc_thread is being hanled +* by ptlrpc_svcpt_stop_threads now +*/ +thread_add_flags (thread,SVC_STOPPED ); +wake_up (&thread->t_ctl_waitq); +spin_unlock (&svcpt->scp_lock); + }else { +list_del (&thread->t_link); +spin_unlock (&svcpt->scp_lock); +OBD_FREE_PTR (thread); + }RETURN (rc); }
Signature · 1 line
1.7.9.5
PATCH 28/48 staging/lustre/crypto: add crc32c module loading to libcfs
In reply to Peng TaoFrom: "Alexander.Boyko" <[email protected]>
This patch add automatically module loading for crc32c when libcfs is starting.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2212
Lustre-change: http://review.whamcloud.com/4372
--- .../lustre/lustre/libcfs/linux/linux-crypto.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/staging/lustre/lustre/libcfs/linux/linux-crypto.c +2 -0
@@ -274,6 +274,8 @@ static int adler32;int cfs_crypto_register (void ) { +request_module ("crc32c" ); + adler32 =cfs_crypto_adler32_register ();/* check all algorithms and do performance test */
Signature · 1 line
1.7.9.5
PATCH 29/48 staging/lustre/mdt: duplicate link names in directory
In reply to Peng TaoFrom: Andreas Dilger <[email protected]>
When creating a hard link to a file, the MDT/MDD/OSD code does not verify whether the target link name already exists in the directory. The ZFS ZAP code checks for duplicate entries. The add_dirent_to_buf() function in ldiskfs only checks entries for duplicates while it is traversing the leaf block looking for free space. Even if it scanned the whole leaf block, this would not work for non-htree directories since there is no guarantee that the name is being inserted into the same leaf block.
To fix this, link should check target object doesn't exist as other creat operations.
Add sanity.sh test_31o with multiple threads racing to link a new name into the directory, while ensuring that there is a free entry in the leaf block that is large enough to hold the duplicate name. This needs to be racy, because otherwise the client VFS will see the existing name and not send the RPC to the MDS, hiding the bug.
Add DLDLMRES/PLDLMRES macros for printing the whole lock resource name (including the name hash) in LDLM_DEBUG() messages in a format similar to DFID/PFID so they can be found in debug logs more easily.
The patch pickes client side change of the original patch, which only contains the DLM printk part.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2901
Lustre-change: http://review.whamcloud.com/6591
--- .../lustre/lustre/include/lustre/lustre_idl.h | 4 + drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 145 ++++++++++---------- 2 files changed, 76 insertions(+), 73 deletions(-)
drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +4 -0
@@ -2690,6 +2690,10 @@ struct ldlm_res_id {__u64 name[RES_NAME_SIZE ]; }; +#define DLDLMRES "["LPX64":"LPX64":"LPX64"]."LPX64i +#define PLDLMRES (res) (res)->lr_name.name[0], (res)->lr_name.name[1], \ + (res)->lr_name.name[2], (res)->lr_name.name[3] +extern void lustre_swab_ldlm_res_id (struct ldlm_res_id *id); static inline int ldlm_res_eq(const struct ldlm_res_id *res0,
drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +72 -73
163 lines not on this page · raw message
Signature · 1 line
1.7.9.5
PATCH 30/48 staging/lustre/llite: call simple_setattr() from ll_md_setattr()
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
This partially reverts the change from "LU-2482 layout: introduce new layout for released files" by calling simple_setattr() from ll_md_setattr() without ATTR_SIZE set. Doing so avoids failed assertions in osc_page_delete(). Disable truncates on released files and modify sanity 229 accordingly.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3448
Lustre-change: http://review.whamcloud.com/6643
--- drivers/staging/lustre/lustre/llite/llite_lib.c | 18 ++++++++---------- drivers/staging/lustre/lustre/lov/lov_io.c | 2 +- 2 files changed, 9 insertions(+), 11 deletions(-)
drivers/staging/lustre/lustre/llite/llite_lib.c +8 -10
@@ -1282,6 +1282,13 @@ int ll_md_setattr(struct dentry *dentry, struct md_op_data *op_data,RETURN (rc); } + ia_valid = op_data->op_attr.ia_valid; +/* inode size will be in ll_setattr_ost, can't do it now since dirty +* cache is not cleared yet. */ + op_data->op_attr.ia_valid &= ~(TIMES_SET_FLAGS |ATTR_SIZE ); + rc =simple_setattr (dentry, &op_data->op_attr); + op_data->op_attr.ia_valid = ia_valid; +/* Extract epoch data if obtained. */ op_data->op_handle = md.body->handle; op_data->op_ioepoch = md.body->ioepoch;
@@ -1364,7 +1371,6 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr)struct ll_inode_info *lli =ll_i2info (inode);struct md_op_data *op_data =NULL ;struct md_open_data *mod =NULL ; -unsigned int valid = attr->ia_valid;int rc =0 , rc1 =0 ;ENTRY ;
@@ -1476,13 +1482,6 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr) rc =ll_setattr_ost (inode, attr);EXIT ; out: -if (rc ==0 ) { -/* Update inode attribute after dirty cache is cleaned -* by truncating OST objects. */ - attr->ia_valid |=ATTR_FORCE ; - rc =simple_setattr (dentry, attr); -LASSERT (rc ==0 ); - }if (op_data) {if (op_data->op_ioepoch) { rc1 =ll_setattr_done_writing (inode, op_data, mod);
@@ -1501,8 +1500,7 @@ out:ll_stats_ops_tally (ll_i2sbi (inode), (attr->ia_valid &ATTR_SIZE ) ?LPROC_LL_TRUNC :LPROC_LL_SETATTR ,1 ); - attr->ia_valid = valid; -RETURN (rc); +return rc; }int ll_setattr (struct dentry *de,struct iattr *attr)
drivers/staging/lustre/lustre/lov/lov_io.c +1 -1
@@ -980,9 +980,9 @@ int lov_io_init_released(const struct lu_env *env, struct cl_object *obj,LASSERTF (0 ,"invalid type %d\n" , io->ci_type);case CIT_MISC :case CIT_FSYNC : -case CIT_SETATTR : result =+1 ;break ; +case CIT_SETATTR :case CIT_READ :case CIT_WRITE :case CIT_FAULT :
Signature · 1 line
1.7.9.5
PATCH 31/48 staging/lustre/ldlm: Fix flock deadlock detection race
In reply to Peng TaoFrom: Andriy Skulysh <[email protected]>
Deadlock isn't detected if 2 threads are trying to grant 2 locks which deadlock on each other. They call ldlm_flock_deadlock() simultaneously and deadlock ins't detected.
The soulition is to add lock to blocking list before calling ldlm_flock_deadlock()
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1602
Lustre-change: http://review.whamcloud.com/3277
--- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
drivers/staging/lustre/lustre/ldlm/ldlm_flock.c +10 -6
@@ -198,6 +198,7 @@ ldlm_flock_deadlock(struct ldlm_lock *req, struct ldlm_lock *bl_lock)if (lock ==NULL )break ; +LASSERT (req != lock); flock = &lock->l_policy_data.l_flock;LASSERT (flock->owner == bl_owner); bl_owner = flock->blocking_owner;
@@ -329,18 +330,21 @@ reprocess:RETURN (LDLM_ITER_STOP ); } -if (ldlm_flock_deadlock (req, lock)) { -ldlm_flock_destroy (req, mode, *flags); - *err = -EDEADLK ; -RETURN (LDLM_ITER_STOP ); - } - +/* add lock to blocking list before deadlock +* check to prevent race */ rc =ldlm_flock_blocking_link (req, lock);if (rc) {ldlm_flock_destroy (req, mode, *flags); *err = rc;RETURN (LDLM_ITER_STOP ); } +if (ldlm_flock_deadlock (req, lock)) { +ldlm_flock_blocking_unlink (req); +ldlm_flock_destroy (req, mode, *flags); + *err = -EDEADLK ; +RETURN (LDLM_ITER_STOP ); + } +ldlm_resource_add_lock (res, &res->lr_waiting, req); *flags |=LDLM_FL_BLOCK_GRANTED ;RETURN (LDLM_ITER_STOP );
Signature · 1 line
1.7.9.5
PATCH 32/48 staging/lustre/lnet: remove empty file lnet/lnet/api-errno.c
In reply to Peng TaoFrom: Peng Tao <[email protected]>
The file is empty. We can just remove it.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2335
Lustre-change: http://review.whamcloud.com/5880
--- drivers/staging/lustre/lnet/lnet/Makefile | 2 +- drivers/staging/lustre/lnet/lnet/api-errno.c | 39 -------------------------- 2 files changed, 1 insertion(+), 40 deletions(-) delete mode 100644 drivers/staging/lustre/lnet/lnet/api-errno.c
drivers/staging/lustre/lnet/lnet/Makefile +1 -1
@@ -1,6 +1,6 @@ obj-$(CONFIG_LNET) += lnet.o -lnet-y := api-errno.o api-ni.o config.o lib-me.o lib-msg.o lib-eq.o \ +lnet-y := api-ni.o config.o lib-me.o lib-msg.o lib-eq.o \ lib-md.o lib-ptl.o lib-move.o module.o lo.o router.o \ router_proc.o acceptor.o peer.o
drivers/staging/lustre/lnet/lnet/api-errno.c deleted +0 -39
@@ -1,39 +0,0 @@ -/* -* GPL HEADER START -* -* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. -* -* This program is free software; you can redistribute it and/or modify -* it under the terms of the GNU General Public License version 2 only, -* as published by the Free Software Foundation. -* -* This program is distributed in the hope that it will be useful, but -* WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -* General Public License version 2 for more details (a copy is included -* in the LICENSE file that accompanied this code). -* -* You should have received a copy of the GNU General Public License -* version 2 along with this program; If not, see -* http://www.sun.com/software/products/lustre/docs/GPLv2.pdf -* -* Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, -* CA 95054 USA or visit www.sun.com if you need additional information or -* have any questions. -* -* GPL HEADER END -*/ -/* -* Copyright (c) 2003, 2010, Oracle and/or its affiliates. All rights reserved. -* Use is subject to license terms. -*/ -/* -* This file is part of Lustre, http://www.lustre.org/ -* Lustre is a trademark of Sun Microsystems, Inc. -* -* lnet/lnet/api-errno.c -* -* Instantiate the string table of errors -*/ - -/* If you change these, you must update the number table in portals/errno.h */
Signature · 1 line
1.7.9.5
PATCH 33/48 staging/lustre/fld: prepare FLD module for client server split
In reply to Peng TaoFrom: Liu Xuezhao <[email protected]>
Split FLD server from client, fld_{handler,index}.c are not compliled unless server support is enabled. Do not include dt_object.h or lustre_mdt.h in lustre_fld.h and fix the minor breakages caused by this elsewhere. Generally cleanup includes in lustre/fld.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1330
Lustre-change: http://review.whamcloud.com/2675
--- drivers/staging/lustre/lustre/fld/Makefile | 2 +- drivers/staging/lustre/lustre/fld/fld_handler.c | 447 -------------------- drivers/staging/lustre/lustre/fld/fld_index.c | 426 ------------------- drivers/staging/lustre/lustre/fld/fld_internal.h | 29 -- drivers/staging/lustre/lustre/fld/fld_request.c | 40 +- drivers/staging/lustre/lustre/fld/lproc_fld.c | 205 +-------- drivers/staging/lustre/lustre/include/lustre_fld.h | 41 -- drivers/staging/lustre/lustre/include/lustre_mdt.h | 84 ---- .../staging/lustre/lustre/obdecho/echo_client.c | 1 + 9 files changed, 37 insertions(+), 1238 deletions(-) delete mode 100644 drivers/staging/lustre/lustre/fld/fld_handler.c delete mode 100644 drivers/staging/lustre/lustre/fld/fld_index.c delete mode 100644 drivers/staging/lustre/lustre/include/lustre_mdt.h
drivers/staging/lustre/lustre/fld/Makefile +1 -1
@@ -1,5 +1,5 @@ obj-$(CONFIG_LUSTRE_FS) += fld.o -fld-y := fld_handler.o fld_request.o fld_cache.o fld_index.o lproc_fld.o +fld-y := fld_request.o fld_cache.o lproc_fld.o ccflags-y := -I$(src)/../include
drivers/staging/lustre/lustre/fld/fld_handler.c deleted +0 -447
447 lines not on this page · raw message
drivers/staging/lustre/lustre/fld/fld_index.c deleted +0 -426
426 lines not on this page · raw message
drivers/staging/lustre/lustre/fld/fld_internal.h +0 -29
@@ -139,38 +139,10 @@ enum {extern struct lu_fld_hash fld_hash[]; - -struct fld_thread_info { -struct req_capsule *fti_pill; -__u64 fti_key; -struct lu_seq_range fti_rec; -struct lu_seq_range fti_lrange; -struct lu_seq_range fti_irange; -}; - -extern struct lu_context_key fld_thread_key; - -int fld_index_init (const struct lu_env *env,struct lu_server_fld *fld, -struct dt_device *dt); - -void fld_index_fini (const struct lu_env *env,struct lu_server_fld *fld); - -int fld_declare_index_create (const struct lu_env *env, -struct lu_server_fld *fld, -const struct lu_seq_range *new, -struct thandle *th); - -int fld_index_create (const struct lu_env *env,struct lu_server_fld *fld, -const struct lu_seq_range *new,struct thandle *th); - -int fld_index_lookup (const struct lu_env *env,struct lu_server_fld *fld, -seqno_t seq,struct lu_seq_range *range); -int fld_client_rpc (struct obd_export *exp,struct lu_seq_range *range,__u32 fld_op);#ifdef LPROCFS -extern struct lprocfs_vars fld_server_proc_list[];extern struct lprocfs_vars fld_client_proc_list[];#endif
@@ -219,5 +191,4 @@ fld_target_name(struct lu_fld_target *tar) }extern proc_dir_entry_t *fld_type_proc_dir; -extern struct file_operations fld_proc_seq_fops;#endif /* __FLD_INTERNAL_H */
drivers/staging/lustre/lustre/fld/fld_request.c +34 -6
@@ -60,6 +60,8 @@#include <lustre_mdc.h> #include "fld_internal.h" +struct lu_context_key fld_thread_key; +/* TODO: these 3 functions are copies of flow-control code from mdc_lib.c * It should be common thing. The same about mdc RPC lock */ static int fld_req_avail (struct client_obd *cli,struct mdc_cache_waiter *mcw)
@@ -280,6 +282,8 @@ int fld_client_del_target(struct lu_client_fld *fld, __u64 idx)EXPORT_SYMBOL (fld_client_del_target);#ifdef LPROCFS +proc_dir_entry_t *fld_type_proc_dir =NULL ; +static int fld_client_proc_init (struct lu_client_fld *fld) {int rc;
@@ -496,12 +500,7 @@ int fld_client_lookup(struct lu_client_fld *fld, seqno_t seq, mdsno_t *mds, res.lsr_start = seq;fld_range_set_type (&res, flags); -if (target->ft_srv !=NULL ) { -LASSERT (env !=NULL ); - rc =fld_server_lookup (env, target->ft_srv, seq, &res); - }else { - rc =fld_client_rpc (target->ft_exp, &res,FLD_LOOKUP ); - } + rc =fld_client_rpc (target->ft_exp, &res,FLD_LOOKUP );if (rc ==0 ) { *mds = res.lsr_index;
@@ -517,3 +516,32 @@ void fld_client_flush(struct lu_client_fld *fld)fld_cache_flush (fld->lcf_cache); }EXPORT_SYMBOL (fld_client_flush); + +static int __init fld_mod_init (void ) +{ + fld_type_proc_dir =lprocfs_register (LUSTRE_FLD_NAME , + proc_lustre_root, +NULL ,NULL ); +if (IS_ERR (fld_type_proc_dir)) +return PTR_ERR (fld_type_proc_dir); + +LU_CONTEXT_KEY_INIT (&fld_thread_key); +lu_context_key_register (&fld_thread_key); +return 0 ; +} + +static void __exit fld_mod_exit(void) +{ +lu_context_key_degister (&fld_thread_key); +if (fld_type_proc_dir !=NULL && !IS_ERR (fld_type_proc_dir)) { +lprocfs_remove (&fld_type_proc_dir); + fld_type_proc_dir =NULL ; + } +} + +MODULE_AUTHOR ("Sun Microsystems, Inc. <http://www.lustre.org/>" ); +MODULE_DESCRIPTION ("Lustre FLD" ); +MODULE_LICENSE ("GPL" ); + +module_init (fld_mod_init) +module_exit (fld_mod_exit)
drivers/staging/lustre/lustre/fld/lproc_fld.c +1 -204
214 lines not on this page · raw message
drivers/staging/lustre/lustre/include/lustre_fld.h +0 -41
56 lines not on this page · raw message
drivers/staging/lustre/lustre/include/lustre_mdt.h deleted +0 -84
84 lines not on this page · raw message
drivers/staging/lustre/lustre/obdecho/echo_client.c +1 -0
@@ -43,6 +43,7 @@#include <lustre_debug.h> #include <lprocfs_status.h> #include <cl_object.h> +#include <md_object.h> #include <lustre_fid.h> #include <lustre_acl.h> #include <lustre_net.h>
Signature · 1 line
1.7.9.5
PATCH 34/48 staging/lustre/lmv: support DNE with HSM.
In reply to Peng TaoFrom: Thomas Leibovici <[email protected]>
Send HSM requests to the appropriate MDT. Split lists of fids of HSM actions into one list per MDT.
Move kuc registration/unregistration from MDC to LMV as this is not MDT related.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3365
Lustre-change: http://review.whamcloud.com/6714
--- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 200 ++++++++++++++++++++++- drivers/staging/lustre/lustre/mdc/mdc_request.c | 18 +- 2 files changed, 205 insertions(+), 13 deletions(-)
drivers/staging/lustre/lustre/lmv/lmv_obd.c +199 -1
@@ -44,6 +44,7 @@#include <asm/div64.h> #include <linux/seq_file.h> #include <linux/namei.h> +#include <asm/uaccess.h> #include <lustre/lustre_idl.h> #include <obd_support.h>
@@ -783,6 +784,125 @@ out_fid2path:RETURN (rc); } +static int lmv_hsm_req_count (struct lmv_obd *lmv, +const struct hsm_user_request *hur, +const struct lmv_tgt_desc *tgt_mds) +{ +int i, nr =0 ; +struct lmv_tgt_desc *curr_tgt; + +/* count how many requests must be sent to the given target */ +for (i =0 ; i < hur->hur_request.hr_itemcount; i++) { + curr_tgt =lmv_find_target (lmv, &hur->hur_user_item[i].hui_fid); +if (obd_uuid_equals (&curr_tgt->ltd_uuid, &tgt_mds->ltd_uuid)) + nr++; + } +return nr; +} + +static void lmv_hsm_req_build (struct lmv_obd *lmv, +struct hsm_user_request *hur_in, +const struct lmv_tgt_desc *tgt_mds, +struct hsm_user_request *hur_out) +{ +int i, nr_out; +struct lmv_tgt_desc *curr_tgt; + +/* build the hsm_user_request for the given target */ + hur_out->hur_request = hur_in->hur_request; + nr_out =0 ; +for (i =0 ; i < hur_in->hur_request.hr_itemcount; i++) { + curr_tgt =lmv_find_target (lmv, + &hur_in->hur_user_item[i].hui_fid); +if (obd_uuid_equals (&curr_tgt->ltd_uuid, &tgt_mds->ltd_uuid)) { + hur_out->hur_user_item[nr_out] = + hur_in->hur_user_item[i]; + nr_out++; + } + } + hur_out->hur_request.hr_itemcount = nr_out; +memcpy (hur_data (hur_out),hur_data (hur_in), + hur_in->hur_request.hr_data_len); +} + +static int lmv_hsm_ct_unregister (struct lmv_obd *lmv,unsigned int cmd,int len, +struct lustre_kernelcomm *lk,void *uarg) +{ +int i, rc =0 ; +ENTRY ; + +/* unregister request (call from llapi_hsm_copytool_fini) */ +for (i =0 ; i < lmv->desc.ld_tgt_count; i++) { +/* best effort: try to clean as much as possible +* (continue on error) */ +obd_iocontrol (cmd, lmv->tgts[i]->ltd_exp, len, lk, uarg); + } + +/* Whatever the result, remove copytool from kuc groups. +* Unreached coordinators will get EPIPE on next requests +* and will unregister automatically. +*/ + rc =libcfs_kkuc_group_rem (lk->lk_uid, lk->lk_group); +RETURN (rc); +} + +static int lmv_hsm_ct_register (struct lmv_obd *lmv,unsigned int cmd,int len, +struct lustre_kernelcomm *lk,void *uarg) +{ +struct file *filp; +int i, j, err; +int rc =0 ; +bool any_set = false; +ENTRY ; + +/* All or nothing: try to register to all MDS. +* In case of failure, unregister from previous MDS, +* except if it because of inactive target. */ +for (i =0 ; i < lmv->desc.ld_tgt_count; i++) { + err =obd_iocontrol (cmd, lmv->tgts[i]->ltd_exp, + len, lk, uarg); +if (err) { +if (lmv->tgts[i]->ltd_active) { +/* permanent error */ +CERROR ("error: iocontrol MDC %s on MDT" +"idx %d cmd %x: err = %d\n" , + lmv->tgts[i]->ltd_uuid.uuid, + i, cmd, err); + rc = err; + lk->lk_flags |=LK_FLG_STOP ; +/* unregister from previous MDS */ +for (j =0 ; j < i; j++) +obd_iocontrol (cmd, + lmv->tgts[j]->ltd_exp, + len, lk, uarg); +RETURN (rc); + } +/* else: transient error. +* kuc will register to the missing MDT +* when it is back */ + }else { + any_set = true; + } + } + +if (!any_set) +/* no registration done: return error */ +RETURN (-ENOTCONN ); + +/* at least one registration done, with no failure */ + filp =fget (lk->lk_wfd); +if (filp ==NULL ) { +RETURN (-EBADF ); + } + rc =libcfs_kkuc_group_add (filp, lk->lk_uid, lk->lk_group, lk->lk_data); +if (rc !=0 && filp !=NULL ) +fput (filp); +RETURN (rc); +} + + + +static int lmv_iocontrol (unsigned int cmd,struct obd_export *exp,int len,void *karg,void *uarg) {
@@ -908,7 +1028,77 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp, }case LL_IOC_HSM_STATE_GET :case LL_IOC_HSM_STATE_SET : -case LL_IOC_HSM_ACTION : +case LL_IOC_HSM_ACTION : { +struct md_op_data *op_data = karg; +struct lmv_tgt_desc *tgt; + + tgt =lmv_find_target (lmv, &op_data->op_fid1); +if (IS_ERR (tgt)) +RETURN (PTR_ERR (tgt)); + +if (tgt->ltd_exp ==NULL ) +RETURN (-EINVAL ); + + rc =obd_iocontrol (cmd, tgt->ltd_exp, len, karg, uarg); +break ; + } +case LL_IOC_HSM_PROGRESS : { +const struct hsm_progress_kernel *hpk = karg; +struct lmv_tgt_desc *tgt; + + tgt =lmv_find_target (lmv, &hpk->hpk_fid); +if (IS_ERR (tgt)) +RETURN (PTR_ERR (tgt)); + rc =obd_iocontrol (cmd, tgt->ltd_exp, len, karg, uarg); +break ; + } +case LL_IOC_HSM_REQUEST : { +struct hsm_user_request *hur = karg; +struct lmv_tgt_desc *tgt; +unsigned int reqcount = hur->hur_request.hr_itemcount; + +if (reqcount ==0 ) +RETURN (0 ); + +/* if the request is about a single fid +* or if there is a single MDS, no need to split +* the request. */ +if (reqcount ==1 || count ==1 ) { + tgt =lmv_find_target (lmv, + &hur->hur_user_item[0 ].hui_fid); +if (IS_ERR (tgt)) +RETURN (PTR_ERR (tgt)); + rc =obd_iocontrol (cmd, tgt->ltd_exp, len, karg, uarg); + }else { +/* split fid list to their respective MDS */ +for (i =0 ; i < count; i++) { +unsigned int nr, reqlen; +int rc1; +struct hsm_user_request *req; + + nr =lmv_hsm_req_count (lmv, hur, lmv->tgts[i]); +if (nr ==0 )/* nothing for this MDS */ +continue ; + +/* build a request with fids for this MDS */ + reqlen = offsetof(typeof(*hur), + hur_user_item[nr]) + + hur->hur_request.hr_data_len; +OBD_ALLOC_LARGE (req, reqlen); +if (req ==NULL ) +RETURN (-ENOMEM ); + +lmv_hsm_req_build (lmv, hur, lmv->tgts[i], req); + + rc1 =obd_iocontrol (cmd, lmv->tgts[i]->ltd_exp, + reqlen, req, uarg); +if (rc1 !=0 && rc ==0 ) + rc = rc1; +OBD_FREE_LARGE (req, reqlen); + } + } +break ; + }case LL_IOC_LOV_SWAP_LAYOUTS : {struct md_op_data *op_data = karg;struct lmv_tgt_desc *tgt1, *tgt2;
@@ -931,6 +1121,14 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp, rc =obd_iocontrol (cmd, tgt1->ltd_exp, len, karg, uarg);break ; } +case LL_IOC_HSM_CT_START : { +struct lustre_kernelcomm *lk = karg; +if (lk->lk_flags &LK_FLG_STOP ) + rc =lmv_hsm_ct_unregister (lmv, cmd, len, lk, uarg); +else + rc =lmv_hsm_ct_register (lmv, cmd, len, lk, uarg); +break ; + }default :for (i =0 ; i < count; i++) {struct obd_device *mdc_obd;
drivers/staging/lustre/lustre/mdc/mdc_request.c +6 -12
@@ -1771,6 +1771,9 @@ static int mdc_iocontrol(unsigned int cmd, struct obd_export *exp, int len,GOTO (out, rc);case LL_IOC_HSM_CT_START : rc =mdc_ioc_hsm_ct_start (exp, karg); +/* ignore if it was already registered on this MDS. */ +if (rc == -EEXIST ) + rc =0 ;GOTO (out, rc);case LL_IOC_HSM_PROGRESS : rc =mdc_ioc_hsm_progress (exp, karg);
@@ -1988,19 +1991,10 @@ static int mdc_ioc_hsm_ct_start(struct obd_export *exp, lk->lk_uid, lk->lk_group, lk->lk_flags);if (lk->lk_flags &LK_FLG_STOP ) { - rc =libcfs_kkuc_group_rem (lk->lk_uid, lk->lk_group);/* Unregister with the coordinator */ -if (rc ==0 ) - rc =mdc_ioc_hsm_ct_unregister (imp); + rc =mdc_ioc_hsm_ct_unregister (imp); }else { -struct file *fp =fget (lk->lk_wfd); - - rc =libcfs_kkuc_group_add (fp, lk->lk_uid, lk->lk_group, - lk->lk_data); -if (rc && fp) -fput (fp); -if (rc ==0 ) - rc =mdc_ioc_hsm_ct_register (imp, archive); + rc =mdc_ioc_hsm_ct_register (imp, archive); }return rc;
@@ -2325,7 +2319,7 @@ static int mdc_import_event(struct obd_device *obd, struct obd_import *imp, }case IMP_EVENT_ACTIVE : rc =obd_notify_observer (obd, obd,OBD_NOTIFY_ACTIVE ,NULL ); -/* restore re-establish kuc registration after reconnecting */ +/* redo the kuc registration after reconnecting */ if (rc ==0 ) rc =mdc_kuc_reregister (imp);break ;
Signature · 1 line
1.7.9.5
PATCH 35/48 staging/lustre/lmv: support DNE with HSM.
In reply to Peng TaoFrom: Thomas Leibovici <[email protected]>
Send HSM requests to the appropriate MDT. Split lists of fids of HSM actions into one list per MDT.
Move kuc registration/unregistration from MDC to LMV as this is not MDT related.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3365
Lustre-change: http://review.whamcloud.com/6714
--- drivers/staging/lustre/lustre/llite/llite_mmap.c | 36 +++++++++++++--------- 1 file changed, 21 insertions(+), 15 deletions(-)
drivers/staging/lustre/lustre/llite/llite_mmap.c +21 -15
@@ -107,11 +107,12 @@ struct cl_io *ll_fault_io_init(struct vm_area_struct *vma,struct cl_env_nest *nest, pgoff_t index, unsigned long *ra_flags) { - struct file *file = vma->vm_file; -struct inode *inode = file->f_dentry->d_inode; -struct cl_io *io; -struct cl_fault_io *fio; -struct lu_env *env; + struct file *file = vma->vm_file; +struct inode *inode = file->f_dentry->d_inode; +struct cl_io *io; +struct cl_fault_io *fio; +struct lu_env *env; +int rc;ENTRY ; *env_ret =NULL ;
@@ -152,17 +153,22 @@ struct cl_io *ll_fault_io_init(struct vm_area_struct *vma,CDEBUG (D_MMAP ,"vm_flags: %lx (%lu %d)\n" , vma->vm_flags, fio->ft_index, fio->ft_executable); -if (cl_io_init (env, io,CIT_FAULT , io->ci_obj) ==0 ) { + rc =cl_io_init (env, io,CIT_FAULT , io->ci_obj); +if (rc ==0 ) {struct ccc_io *cio =ccc_env_io (env);struct ll_file_data *fd =LUSTRE_FPRIVATE (file);LASSERT (cio->cui_cl.cis_io == io); -/* mmap lock must be MANDATORY -* it has to cache pages. */ +/* mmap lock must be MANDATORY it has to cache +* pages. */ io->ci_lockreq =CILR_MANDATORY ; - - cio->cui_fd = fd; + cio->cui_fd = fd; + }else { +LASSERT (rc <0 ); +cl_io_fini (env, io); +cl_env_nested_put (nest, env); + io =ERR_PTR (rc); }return io;
@@ -190,7 +196,7 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage, result = io->ci_result;if (result <0 ) -GOTO (out, result); +GOTO (out_io, result); io->u.ci_fault.ft_mkwrite =1 ; io->u.ci_fault.ft_writable =1 ;
@@ -252,14 +258,14 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage, }EXIT ; -out: +out_io:cl_io_fini (env, io);cl_env_nested_put (&nest, env); - +out:CDEBUG (D_MMAP ,"%s mkwrite with %d\n" , current->comm, result); -LASSERT (ergo (result ==0 ,PageLocked (vmpage))); -return (result); + +return result; }
Signature · 1 line
1.7.9.5
Greg Kroah-Hartman
maintainerRe: [PATCH 35/48] staging/lustre/lmv: support DNE with HSM.
Quoted 15 lines · Peng Tao
From: Thomas Leibovici <[email protected]>
Send HSM requests to the appropriate MDT. Split lists of fids of HSM actions into one list per MDT.
Move kuc registration/unregistration from MDC to LMV as this is not MDT related.Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3365
Lustre-change: http://review.whamcloud.com/6714Signed-off-by Thomas Leibovici <[email protected]>Reviewed-by John L. Hammond <[email protected]>Reviewed-by jacques-Charles Lafoucriere <[email protected]>Reviewed-by Oleg Drokin <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Andreas Dilger <[email protected]>
Quoted 3 lines
--- drivers/staging/lustre/lustre/llite/llite_mmap.c | 36 +++++++++++++--------- 1 file changed, 21 insertions(+), 15 deletions(-)
This has the exact same Subject: as patch 34, which implies that one of these is wrong, so I'll not take either, sorry.
Please fix up and resend.
greg k-h
PATCH 36/48 staging/lustre/obdclass: add obd_target.h
In reply to Peng TaoFrom: Liu Xuezhao <[email protected]>
Move some server-side data structures form obd.h to obd_target.h. To remove some code dependencies use HAVE_SERVER_SUPPORT to protect some llog/fsfilt_* functions.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1330
Lustre-change: http://review.whamcloud.com/2678
--- .../lustre/lustre/include/linux/lustre_fsfilt.h | 10 - .../staging/lustre/lustre/include/lprocfs_status.h | 36 -- drivers/staging/lustre/lustre/include/obd.h | 140 +---- drivers/staging/lustre/lustre/lvfs/lvfs_linux.c | 1 - drivers/staging/lustre/lustre/obdclass/Makefile | 2 +- .../lustre/lustre/obdclass/lprocfs_jobstats.c | 560 -------------------- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 1 - 7 files changed, 3 insertions(+), 747 deletions(-) delete mode 100644 drivers/staging/lustre/lustre/obdclass/lprocfs_jobstats.c
drivers/staging/lustre/lustre/include/linux/lustre_fsfilt.h +0 -10
@@ -145,16 +145,6 @@ static inline int fsfilt_commit(struct obd_device *obd, struct inode *inode,return rc; } -static inline int fsfilt_map_inode_pages (struct obd_device *obd, -struct inode *inode, -struct page **page,int pages, -unsigned long *blocks, -int create,struct mutex *mutex) -{ -return obd->obd_fsops->fs_map_inode_pages (inode, page, pages, blocks, - create, mutex); -} -static inline int fsfilt_read_record (struct obd_device *obd,struct file *file,void *buf,loff_t size,loff_t *offs) {
drivers/staging/lustre/lustre/include/lprocfs_status.h +0 -36
@@ -370,18 +370,6 @@ static inline void s2dhms(struct dhms *ts, time_t secs)#define JOBSTATS_DISABLE "disable"#define JOBSTATS_PROCNAME_UID "procname_uid" -typedef void (*cntr_init_callback )(struct lprocfs_stats *stats); - -struct obd_job_stats { -cfs_hash_t *ojs_hash; -struct list_head ojs_list; -rwlock_t ojs_lock;/* protect the obj_list */ -cntr_init_callback ojs_cntr_init_fn; -int ojs_cntr_num; -int ojs_cleanup_interval; -time_t ojs_last_cleanup; -}; -#ifdef LPROCFS extern int lprocfs_stats_alloc_one(struct lprocfs_stats *stats,
@@ -748,16 +736,6 @@ struct file_operations name##_fops = { \ .release = lprocfs_single_release, \ }; -/* lprocfs_jobstats.c */ -int lprocfs_job_stats_log (struct obd_device *obd,char *jobid, -int event,long amount); -void lprocfs_job_stats_fini (struct obd_device *obd); -int lprocfs_job_stats_init (struct obd_device *obd,int cntr_num, -cntr_init_callback fn); -int lprocfs_rd_job_interval (struct seq_file *m,void *data); -int lprocfs_wr_job_interval (struct file *file,const char *buffer, -unsigned long count,void *data); -/* lproc_ptlrpc.c */ struct ptlrpc_request ;extern void target_print_req (void *seq_file,struct ptlrpc_request *req);
@@ -1021,20 +999,6 @@ __u64 lprocfs_stats_collector(struct lprocfs_stats *stats, int idx,#define LPROC_SEQ_FOPS_RW_TYPE (name, type)#define LPROC_SEQ_FOPS_WR_ONLY (name, type) -/* lprocfs_jobstats.c */ -static inline -int lprocfs_job_stats_log (struct obd_device *obd,char *jobid,int event, -long amount) -{return 0 ; } -static inline -void lprocfs_job_stats_fini (struct obd_device *obd) -{return ; } -static inline -int lprocfs_job_stats_init (struct obd_device *obd,int cntr_num, -cntr_init_callback fn) -{return 0 ; } - -/* lproc_ptlrpc.c */ #define target_print_req NULL
drivers/staging/lustre/lustre/include/obd.h +2 -138
178 lines not on this page · raw message
drivers/staging/lustre/lustre/lvfs/lvfs_linux.c +0 -1
@@ -48,7 +48,6 @@#include <linux/quotaops.h> #include <linux/version.h> #include <linux/libcfs/libcfs.h> -#include <lustre_fsfilt.h> #include <obd.h> #include <linux/module.h> #include <linux/init.h>
drivers/staging/lustre/lustre/obdclass/Makefile +1 -1
@@ -3,7 +3,7 @@ obj-$(CONFIG_LUSTRE_FS) += obdclass.o llog_test.o obdclass-y := linux/linux-module.o linux/linux-obdo.o linux/linux-sysctl.o \ llog.o llog_cat.o llog_obd.o llog_swab.o class_obd.o debug.o \ genops.o uuid.o llog_ioctl.o lprocfs_status.o \ - lprocfs_jobstats.o lustre_handles.o lustre_peer.o llog_osd.o \ + lustre_handles.o lustre_peer.o llog_osd.o \ local_storage.o statfs_pack.o obdo.o obd_config.o obd_mount.o\ mea.o lu_object.o dt_object.o capa.o cl_object.o \ cl_page.o cl_lock.o cl_io.o acl.o idmap.o lu_ucred.o
drivers/staging/lustre/lustre/obdclass/lprocfs_jobstats.c deleted +0 -560
560 lines not on this page · raw message
drivers/staging/lustre/lustre/obdclass/obd_mount.c +0 -1
@@ -47,7 +47,6 @@#include <obd.h> #include <lvfs.h> -#include <lustre_fsfilt.h> #include <obd_class.h> #include <lustre/lustre_user.h> #include <linux/version.h>
Signature · 1 line
1.7.9.5
Greg Kroah-Hartman
maintainerRe: [PATCH 36/48] staging/lustre/obdclass: add obd_target.h
Quoted 15 lines · Peng Tao
From: Liu Xuezhao <[email protected]>
Move some server-side data structures form
obd.htoobd_target.h. To remove some code dependencies useHAVE_SERVER_SUPPORTto protect some llog/fsfilt_* functions.Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1330
Lustre-change: http://review.whamcloud.com/2678Signed-off-by Liu Xuezhao <[email protected]>Signed-off-by John L. Hammond <[email protected]>Reviewed-by Andreas Dilger <[email protected]>Reviewed-by Peng Tao <[email protected]>Reviewed-by Mike Pershin <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Andreas Dilger <[email protected]>
Quoted 6 lines
--- .../lustre/lustre/include/linux/lustre_fsfilt.h | 10 - .../staging/lustre/lustre/include/lprocfs_status.h | 36 -- drivers/staging/lustre/lustre/include/obd.h | 140 +---- drivers/staging/lustre/lustre/lvfs/lvfs_linux.c | 1 - drivers/staging/lustre/lustre/obdclass/Makefile | 2 +-
This patch has too much fuzz to apply properly, please refresh and resend.
thanks,
greg k-h
PATCH 37/48 staging/lustre/procfs: return -ENOMEM from lprocfs_register()
In reply to Peng TaoFrom: "John L. Hammond" <[email protected]>
In lprocfs_register(), if proc_mkdir() fails then return ERR_PTR(-ENOMEM) rather than NULL and hold _lprocfs_mutex for the whole function. In lprocfs_remove_nolock() return early if the entry is an error pointer. Improve error handling around lprocfs_register() in a few spots.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2650
Lustre-change: http://review.whamcloud.com/5161
--- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 1 + drivers/staging/lustre/lustre/llite/super25.c | 2 +- drivers/staging/lustre/lustre/lov/lov_obd.c | 6 ++--- .../lustre/lustre/obdclass/linux/linux-module.c | 9 ++++++- .../lustre/lustre/obdclass/lprocfs_status.c | 28 ++++++++++++-------- .../staging/lustre/lustre/ptlrpc/gss/lproc_gss.c | 6 +++-- 6 files changed, 34 insertions(+), 18 deletions(-)
drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +1 -0
@@ -750,6 +750,7 @@ static int ldlm_pool_proc_init(struct ldlm_pool *pl)if (IS_ERR (pl->pl_proc_dir)) {CERROR ("LProcFS failed in ldlm-pool-init\n" ); rc =PTR_ERR (pl->pl_proc_dir); + pl->pl_proc_dir =NULL ;GOTO (out_free_name, rc); }
drivers/staging/lustre/lustre/llite/super25.c +1 -1
@@ -214,7 +214,7 @@ static void __exit exit_lustre_lite(void) ll_remote_perm_cachep =NULL ;kmem_cache_destroy (ll_file_data_slab); -if (proc_lustre_fs_root) +if (proc_lustre_fs_root && !IS_ERR (proc_lustre_fs_root))lprocfs_remove (&proc_lustre_fs_root); }
drivers/staging/lustre/lustre/lov/lov_obd.c +3 -3
@@ -836,11 +836,11 @@ int lov_setup(struct obd_device *obd, struct lustre_cfg *lcfg)lprocfs_obd_setup (obd, lvars.obd_vars);#ifdef LPROCFS { -int rc; +int rc1; - rc =lprocfs_seq_create (obd->obd_proc_entry,"target_obd" , + rc1 =lprocfs_seq_create (obd->obd_proc_entry,"target_obd" ,0444 , &lov_proc_target_fops, obd); -if (rc) +if (rc1)CWARN ("Error adding the target_obd file\n" ); }#endif
drivers/staging/lustre/lustre/obdclass/linux/linux-module.c +8 -1
@@ -385,14 +385,21 @@ struct file_operations obd_device_list_fops = {int class_procfs_init (void ) { -int rc; +int rc =0 ;ENTRY ;obd_sysctl_init (); proc_lustre_root =lprocfs_register ("fs/lustre" ,NULL , lprocfs_base,NULL ); +if (IS_ERR (proc_lustre_root)) { + rc =PTR_ERR (proc_lustre_root); + proc_lustre_root =NULL ; + goto out; + } + rc =lprocfs_seq_create (proc_lustre_root,"devices" ,0444 , &obd_device_list_fops,NULL ); +out:if (rc)CERROR ("error adding /proc/fs/lustre/devices file\n" );RETURN (0 );
drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +17 -11
@@ -179,17 +179,21 @@ struct proc_dir_entry *lprocfs_register(const char *name,struct proc_dir_entry *parent, struct lprocfs_vars *list, void *data) { - struct proc_dir_entry *newchild; - - newchild =proc_mkdir (name, parent); -if (newchild !=NULL && list !=NULL ) { -int rc =lprocfs_add_vars (newchild, list, data); -if (rc) { -lprocfs_remove (&newchild); -return ERR_PTR (rc); + struct proc_dir_entry *entry; + + entry =proc_mkdir (name, parent); +if (entry ==NULL ) +GOTO (out, entry =ERR_PTR (-ENOMEM )); + +if (list !=NULL ) { +int rc =lprocfs_add_vars (entry, list, data); +if (rc !=0 ) { +lprocfs_remove (&entry); + entry =ERR_PTR (rc); } } -return newchild; +out: +return entry; }EXPORT_SYMBOL (lprocfs_register);
@@ -1596,10 +1600,12 @@ int lprocfs_exp_setup(struct obd_export *exp, lnet_nid_t *nid, int *newnid) NULL, NULL);OBD_FREE (buffer,LNET_NIDSTR_SIZE ); -if (new_stat->nid_proc ==NULL ) { +if (IS_ERR (new_stat->nid_proc)) {CERROR ("Error making export directory for nid %s\n" ,libcfs_nid2str (*nid)); -GOTO (destroy_new_ns, rc = -ENOMEM ); + rc =PTR_ERR (new_stat->nid_proc); + new_stat->nid_proc =NULL ; +GOTO (destroy_new_ns, rc); } entry = lprocfs_add_simple(new_stat->nid_proc, "uuid",
drivers/staging/lustre/lustre/ptlrpc/gss/lproc_gss.c +4 -2
@@ -199,15 +199,17 @@ int gss_init_lproc(void) gss_proc_root =lprocfs_register ("gss" , sptlrpc_proc_root, gss_lprocfs_vars,NULL );if (IS_ERR (gss_proc_root)) { + rc =PTR_ERR (gss_proc_root); gss_proc_root =NULL ; -GOTO (err_out, rc =PTR_ERR (gss_proc_root)); +GOTO (err_out, rc); } gss_proc_lk =lprocfs_register ("lgss_keyring" , gss_proc_root, gss_lk_lprocfs_vars,NULL );if (IS_ERR (gss_proc_lk)) { + rc =PTR_ERR (gss_proc_lk); gss_proc_lk =NULL ; -GOTO (err_out, rc =PTR_ERR (gss_proc_root)); +GOTO (err_out, rc); }return 0 ;
Signature · 1 line
1.7.9.5
PATCH 38/48 staging/lustre/lmv: fix duplicate directory entries
In reply to Peng TaoFrom: Ned Bass <[email protected]>
Previously, I accidentally introduced a new way for duplicate directory entries to be returned from readdir(). That patch fails to properly decrement the nlupgs counter when breaking out of the inner-for loop. This accounting error causes an extra iteration of the inner-for loop when processing the next cfs page and a bad ldp_hash_end value is then saved in the lu_dirpage. To fix this, always decrement the nlupgs counter on entry into the inner loop.
Note: this bug only affects architectures with > 4k-sized pages, e.g. PowerPC.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3182
Lustre-change: http://review.whamcloud.com/6405
--- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/staging/lustre/lustre/lmv/lmv_obd.c +2 -1
@@ -2157,7 +2157,7 @@ static void lmv_adjust_dirpages(struct page **pages, int ncfspgs, int nlupgs)__u64 hash_end = dp->ldp_hash_end;__u32 flags = dp->ldp_flags; -for (; nlupgs >1 ; nlupgs--) { +while (--nlupgs >0 ) { ent =lu_dirent_start (dp);for (end_dirent = ent; ent !=NULL ; end_dirent = ent, ent =lu_dirent_next (ent));
@@ -2191,6 +2191,7 @@ static void lmv_adjust_dirpages(struct page **pages, int ncfspgs, int nlupgs) kunmap(pages[i]); } +LASSERTF (nlupgs ==0 ,"left = %d" , nlupgs); }#else #define lmv_adjust_dirpages (pages, ncfspgs, nlupgs) do {} while (0)
Signature · 1 line
1.7.9.5
PATCH 39/48 staging/lustre/obdclass: be more careful processing server name
In reply to Peng TaoFrom: Nathaniel Clark <[email protected]>
Because whole options line gets passed to exclude processing, don't search from end of passed in argument to determine fsname at beginning.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2200
Lustre-change: http://review.whamcloud.com/6197
--- .../staging/lustre/lustre/obdclass/obd_config.c | 11 ++-- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 58 +++++++++++--------- 2 files changed, 40 insertions(+), 29 deletions(-)
drivers/staging/lustre/lustre/obdclass/obd_config.c +7 -4
@@ -1426,10 +1426,13 @@ int class_config_llog_handler(const struct lu_env *env, } -if ((clli->cfg_flags &CFG_F_EXCLUDE ) && - (lcfg->lcfg_command ==LCFG_LOV_ADD_OBD )) -/* Add inactive instead */ - lcfg->lcfg_command =LCFG_LOV_ADD_INA ; +if (clli->cfg_flags &CFG_F_EXCLUDE ) { +CDEBUG (D_CONFIG ,"cmd: %x marked EXCLUDED\n" , + lcfg->lcfg_command); +if (lcfg->lcfg_command ==LCFG_LOV_ADD_OBD ) +/* Add inactive instead */ + lcfg->lcfg_command =LCFG_LOV_ADD_INA ; + }lustre_cfg_bufs_init (&bufs, lcfg);
drivers/staging/lustre/lustre/obdclass/obd_mount.c +33 -25
109 lines not on this page · raw message
Signature · 1 line
1.7.9.5
PATCH 40/48 staging/lustre/llite: return valid fsid for statfs
In reply to Peng TaoFrom: Fan Yong <[email protected]>
Lustre uses 64-bits inode number to identify object on client side. When re-export Lustre via NFS, NFS will detect whether support fsid via statfs(). For the non-support case, it will only recognizes and packs low 32-bits inode number in nfs handle. Such handle cannot be used to locate the object properly.
To avoid patch linux kernel, Lustre client should generate fsid and return it via statfs() to up layer.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2904
Lustre-change: http://review.whamcloud.com/6493
--- .../staging/lustre/lustre/llite/llite_internal.h | 2 ++ drivers/staging/lustre/lustre/llite/llite_lib.c | 6 ++++-- drivers/staging/lustre/lustre/llite/llite_nfs.c | 16 ++++++++++++++++ 3 files changed, 22 insertions(+), 2 deletions(-)
drivers/staging/lustre/lustre/llite/llite_internal.h +2 -0
@@ -505,6 +505,7 @@ struct ll_sb_info {* clustred nfs */ struct rmtacl_ctl_table ll_rct;struct eacl_table ll_et; +__kernel_fsid_t ll_fsid; };#define LL_DEFAULT_MAX_RW_CHUNK (32 * 1024 * 1024)
@@ -833,6 +834,7 @@ char *ll_get_fsname(struct super_block *sb, char *buf, int buflen);/* llite/llite_nfs.c */ extern struct export_operations lustre_export_operations;__u32 get_uuid2int (const char *name,int len); +void get_uuid2fsid (const char *name,int len,__kernel_fsid_t *fsid);struct inode *search_inode_for_lustre (struct super_block *sb,const struct lu_fid *fid);
drivers/staging/lustre/lustre/llite/llite_lib.c +4 -2
@@ -583,8 +583,10 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt,/* s_dev is also used in lt_compare() to compare two fs, but that is * only a node-local comparison. */ uuid =obd_get_uuid (sbi->ll_md_exp); -if (uuid !=NULL ) +if (uuid !=NULL ) { sb->s_dev =get_uuid2int (uuid->uuid,strlen (uuid->uuid)); +get_uuid2fsid (uuid->uuid,strlen (uuid->uuid), &sbi->ll_fsid); + }if (data !=NULL )OBD_FREE_PTR (data);
@@ -1615,7 +1617,7 @@ int ll_statfs(struct dentry *de, struct kstatfs *sfs) sfs->f_blocks = osfs.os_blocks; sfs->f_bfree = osfs.os_bfree; sfs->f_bavail = osfs.os_bavail; - + sfs->f_fsid =ll_s2sbi (sb)->ll_fsid;return 0 ; }
drivers/staging/lustre/lustre/llite/llite_nfs.c +16 -0
@@ -58,6 +58,22 @@ __u32 get_uuid2int(const char *name, int len)return (key0 <<1 ); } +void get_uuid2fsid (const char *name,int len,__kernel_fsid_t *fsid) +{ +__u64 key =0 , key0 =0x12a3fe2d , key1 =0x37abe8f9 ; + +while (len--) { + key = key1 + (key0 ^ (*name++ *7152373 )); +if (key &0x8000000000000000ULL ) + key -=0x7fffffffffffffffULL ; + key1 = key0; + key0 = key; + } + + fsid->val[0 ] = key; + fsid->val[1 ] = key >>32 ; +} +static int ll_nfs_test_inode (struct inode *inode,void *opaque) { return lu_fid_eq(&ll_i2info(inode)->lli_fid,
Signature · 1 line
1.7.9.5
PATCH 41/48 staging/lustre/llite: error of listxattr when buffer is small
In reply to Peng TaoFrom: Keith Mannthey <[email protected]>
According to the standard, listxattr(2) should return -1 and errno should be set to ERANGE if the size of the list buffer is too small to hold the result. However ll_listxattr() will return a value bigger than the size of buffer in some cases.
Let's assume listxattr(2) returns SIZE when it is called with a large enough list buffer. If it's called again with a list buffer whose size is smaller than SIZE but bigger than (SIZE - 12), then listxattr(2) will return SIZE too. This patch fixes the problem. Original patch by Li Xi <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3403
Lustre-change: http://review.whamcloud.com/6463
--- drivers/staging/lustre/lustre/llite/xattr.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/staging/lustre/lustre/llite/xattr.c +6 -1
@@ -564,7 +564,12 @@ ssize_t ll_listxattr(struct dentry *dentry, char *buffer, size_t size)const size_t name_len =sizeof ("lov" ) -1 ;const size_t total_len = prefix_len + name_len +1 ; -if (buffer && (rc + total_len) <= size) { +if (((rc + total_len) > size) && (buffer !=NULL )) { +ptlrpc_req_finished (request); +return -ERANGE ; + } + +if (buffer !=NULL ) { buffer += rc;memcpy (buffer,XATTR_LUSTRE_PREFIX , prefix_len);memcpy (buffer + prefix_len,"lov" , name_len);
Signature · 1 line
1.7.9.5
PATCH 42/48 staging/lustre/llite: Anonymous dentry incorrectly identified as root
In reply to Peng TaoFrom: Patrick Farrell <[email protected]>
When exporting Lustre via NFS on SLES11SP2, this check incorrectly identifies anonymous dentries as root dentries. This is due to a change in SLES11SP2 which makes the d_name.name for anonymous dentries the same as that for root dentries. (Details in LU-3484.)
This changes the check to directly compare the value of the dentry pointer to the root dentry pointer found in the superblock, rather than using the name.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3484
Lustre-change: http://review.whamcloud.com/6726
--- drivers/staging/lustre/lustre/llite/dcache.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/staging/lustre/lustre/llite/dcache.c +4 -3
@@ -295,9 +295,10 @@ void ll_invalidate_aliases(struct inode *inode) dentry->d_name.name, dentry, dentry->d_parent, dentry->d_inode, dentry->d_flags); -if (dentry->d_name.len ==1 && dentry->d_name.name[0 ] =='/' ) { -CERROR ("called on root (?) dentry=%p, inode=%p " -"ino=%lu\n" , dentry, inode, inode->i_ino); +if (unlikely (dentry == dentry->d_sb->s_root)) { +CERROR ("%s: called on root dentry=%p, fid=" DFID "\n" , +ll_get_fsname (dentry->d_sb,NULL ,0 ), + dentry,PFID (ll_inode2fid (inode)));lustre_dump_dentry (dentry,1 );dump_stack (); }
Signature · 1 line
1.7.9.5
PATCH 43/48 staging/lustre/build: fix 'data race condition' issues
In reply to Peng TaoFrom: Sebastien Buisson <[email protected]>
Fix 'data race condition' defects found by Coverity version 6.5.0:
Data race condition (MISSING_LOCK)
Accessing variable without holding lock. Elsewhere, this variable is accessed with lock held.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2744
Lustre-change: http://review.whamcloud.com/6569
--- .../staging/lustre/lustre/include/lustre_import.h | 2 ++ 1 file changed, 2 insertions(+)
drivers/staging/lustre/lustre/include/lustre_import.h +2 -0
@@ -336,9 +336,11 @@ static inline unsigned int at_timeout2est(unsigned int val) }static inline void at_reset (struct adaptive_timeout *at,int val) { +spin_lock (&at->at_lock); at->at_current = val; at->at_worst_ever = val; at->at_worst_time =cfs_time_current_sec (); +spin_unlock (&at->at_lock); }static inline void at_init (struct adaptive_timeout *at,int val,int flags) {memset (at,0 ,sizeof (*at));
Signature · 1 line
1.7.9.5
PATCH 44/48 staging/lustre/build: fix 'data race condition' issues
In reply to Peng TaoFrom: Sebastien Buisson <[email protected]>
Fix 'data race condition' defects found by Coverity version 6.5.0:
Data race condition (MISSING_LOCK)
Accessing variable without holding lock. Elsewhere, this variable is accessed with lock held.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2744
Lustre-change: http://review.whamcloud.com/6571
--- drivers/staging/lustre/lustre/llite/llite_lib.c | 4 ++++ drivers/staging/lustre/lustre/llite/namei.c | 5 ++++- 2 files changed, 8 insertions(+), 1 deletion(-)
drivers/staging/lustre/lustre/llite/llite_lib.c +4 -0
@@ -1190,7 +1190,9 @@ void ll_clear_inode(struct inode *inode)LASSERT (lli->lli_opendir_pid ==0 ); } +spin_lock (&lli->lli_lock);ll_i2info (inode)->lli_flags &= ~LLIF_MDS_SIZE_LOCK ; +spin_unlock (&lli->lli_lock);md_null_inode (sbi->ll_md_exp,ll_inode2fid (inode));LASSERT (!lli->lli_open_fd_write_count);
@@ -1757,7 +1759,9 @@ void ll_update_inode(struct inode *inode, struct lustre_md *md)/* Use old size assignment to avoid * deadlock bz14138 & bz14326 */ i_size_write (inode, body->size); +spin_lock (&lli->lli_lock); lli->lli_flags |=LLIF_MDS_SIZE_LOCK ; +spin_unlock (&lli->lli_lock); }ldlm_lock_decref (&lockh, mode); }
drivers/staging/lustre/lustre/llite/namei.c +4 -1
@@ -275,8 +275,11 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc,CDEBUG (D_INODE ,"invaliding layout %d.\n" , rc); } -if (bits &MDS_INODELOCK_UPDATE ) +if (bits &MDS_INODELOCK_UPDATE ) { +spin_lock (&lli->lli_lock); lli->lli_flags &= ~LLIF_MDS_SIZE_LOCK ; +spin_unlock (&lli->lli_lock); + }if (S_ISDIR (inode->i_mode) && (bits &MDS_INODELOCK_UPDATE )) {
Signature · 1 line
1.7.9.5
Greg Kroah-Hartman
maintainerRe: [PATCH 44/48] staging/lustre/build: fix 'data race condition' issues
Quoted 7 lines · Peng Tao
From: Sebastien Buisson <[email protected]>
Fix 'data race condition' defects found by Coverity version 6.5.0:
Data race condition (MISSING_LOCK)
Accessing variable without holding lock. Elsewhere, this variable is accessed with lock held.
Again duplicate Subject: lines with the previous patch, so I've dropped both.
thanks,
greg k-h
PATCH 45/48 staging/lustre/style: removes obsolete EXPORT_SYMTAB macros v2
In reply to Peng TaoFrom: Keith Mannthey <[email protected]>
EXPORT_SYMTAB is obsolete after 2.4 kernel, this patch removes EXPORT_SYMTAB from the source code again.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1347
Lustre-change: http://review.whamcloud.com/6739
--- drivers/staging/lustre/lustre/libcfs/libcfs_cpu.c | 3 --- drivers/staging/lustre/lustre/libcfs/libcfs_lock.c | 3 --- drivers/staging/lustre/lustre/libcfs/libcfs_mem.c | 3 --- drivers/staging/lustre/lustre/obdclass/llog_osd.c | 4 ---- 4 files changed, 13 deletions(-)
drivers/staging/lustre/lustre/libcfs/libcfs_cpu.c +0 -3
@@ -33,9 +33,6 @@* Author: [email protected] */ -#ifndef EXPORT_SYMTAB -# define EXPORT_SYMTAB -#endif #define DEBUG_SUBSYSTEM S_LNET#include <linux/libcfs/libcfs.h>
drivers/staging/lustre/lustre/libcfs/libcfs_lock.c +0 -3
@@ -30,9 +30,6 @@* Author: [email protected] */ -#ifndef EXPORT_SYMTAB -# define EXPORT_SYMTAB -#endif #define DEBUG_SUBSYSTEM S_LNET#include <linux/libcfs/libcfs.h>
drivers/staging/lustre/lustre/libcfs/libcfs_mem.c +0 -3
@@ -31,9 +31,6 @@* Author: [email protected] */ -#ifndef EXPORT_SYMTAB -# define EXPORT_SYMTAB -#endif #define DEBUG_SUBSYSTEM S_LNET#include <linux/libcfs/libcfs.h>
drivers/staging/lustre/lustre/obdclass/llog_osd.c +0 -4
@@ -41,10 +41,6 @@#define DEBUG_SUBSYSTEM S_LOG -#ifndef EXPORT_SYMTAB -#define EXPORT_SYMTAB -#endif -#include <obd.h> #include <obd_class.h> #include <lustre_fid.h>
Signature · 1 line
1.7.9.5
PATCH 46/48 staging/lustre/mdt: HSM on disk actions record
In reply to Peng TaoFrom: jcl <[email protected]>
HSM coordinator memorizes all actions in a llog This patch implements the methods needed to create update display these records.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3339
Lustre-change: http://review.whamcloud.com/6529
--- .../staging/lustre/lustre/include/lprocfs_status.h | 3 -- .../lustre/lustre/include/lustre/lustre_idl.h | 47 ++++++++++++++++ .../staging/lustre/lustre/include/lustre_disk.h | 56 ++++++++++---------- drivers/staging/lustre/lustre/include/obd.h | 7 +-- drivers/staging/lustre/lustre/obdclass/llog_swab.c | 17 ++++++ 5 files changed, 97 insertions(+), 33 deletions(-)
drivers/staging/lustre/lustre/include/lprocfs_status.h +0 -3
@@ -804,9 +804,6 @@ extern int lprocfs_quota_rd_qs_factor(char *page, char **start, loff_t off,extern int lprocfs_quota_wr_qs_factor (struct file *file,const char *buffer,unsigned long count,void *data); - - -#else /* LPROCFS is not defined */
drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +47 -0
59 lines not on this page · raw message
drivers/staging/lustre/lustre/include/lustre_disk.h +29 -27
70 lines not on this page · raw message
drivers/staging/lustre/lustre/include/obd.h +4 -3
@@ -267,9 +267,10 @@ enum llog_ctxt_id {LLOG_TEST_REPL_CTXT ,LLOG_LOVEA_ORIG_CTXT ,LLOG_LOVEA_REPL_CTXT , -LLOG_CHANGELOG_ORIG_CTXT ,/**< changelog generation on mdd */ -LLOG_CHANGELOG_REPL_CTXT ,/**< changelog access on clients */ -LLOG_CHANGELOG_USER_ORIG_CTXT ,/**< for multiple changelog consumers */ +LLOG_CHANGELOG_ORIG_CTXT ,/**< changelog generation on mdd */ +LLOG_CHANGELOG_REPL_CTXT ,/**< changelog access on clients */ +LLOG_CHANGELOG_USER_ORIG_CTXT ,/**< for multiple changelog consumers */ +LLOG_AGENT_ORIG_CTXT ,/**< agent requests generation on cdt */ LLOG_MAX_CTXTS };
drivers/staging/lustre/lustre/obdclass/llog_swab.c +17 -0
@@ -203,6 +203,23 @@ void lustre_swab_llog_rec(struct llog_rec_hdr *rec)break ; } +case HSM_AGENT_REC : { +struct llog_agent_req_rec *arr = + (struct llog_agent_req_rec *)rec; + +__swab32s (&arr->arr_hai.hai_len); +__swab32s (&arr->arr_hai.hai_action); +lustre_swab_lu_fid (&arr->arr_hai.hai_fid); +lustre_swab_lu_fid (&arr->arr_hai.hai_dfid); +__swab64s (&arr->arr_hai.hai_cookie); +__swab64s (&arr->arr_hai.hai_extent.offset); +__swab64s (&arr->arr_hai.hai_extent.length); +__swab64s (&arr->arr_hai.hai_gid); +/* no swabing for opaque data */ +/* hai_data[0]; */ +break ; + } +case MDS_SETATTR64_REC : { struct llog_setattr64_rec *lsr =
Signature · 1 line
1.7.9.5
PATCH 47/48 staging/lustre/scrub: purge inconsistenct objects after OI scrub
In reply to Peng TaoFrom: Fan Yong <[email protected]>
When OI scrub repaired the found inconsistency, it needs to purge the old object out of cache; otherwise, others may still use those cached stale information.
Original patch adds functions in obdclass that is only used by server. Drop that part. Only merge in error handling change.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3335
Lustre-change: http://review.whamcloud.com/6697
--- drivers/staging/lustre/lustre/obdclass/lu_object.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/staging/lustre/lustre/obdclass/lu_object.c +5 -5
@@ -538,7 +538,7 @@ static struct lu_object *htable_lookup(struct lu_site *s,__u64 ver =cfs_hash_bd_version_get (bd);if (*version == ver) -return NULL ; +return ERR_PTR (-ENOENT ); *version = ver; bkt =cfs_hash_bd_extra_get (s->ls_obj_hash, bd);
@@ -547,7 +547,7 @@ static struct lu_object *htable_lookup(struct lu_site *s, hnode =cfs_hash_bd_peek_locked (s->ls_obj_hash, bd, (void *)f);if (hnode ==NULL ) {lprocfs_counter_incr (s->ls_stats,LU_SS_CACHE_MISS ); -return NULL ; +return ERR_PTR (-ENOENT ); } h = container_of0(hnode, struct lu_object_header, loh_hash);
@@ -651,7 +651,7 @@ static struct lu_object *lu_object_find_try(const struct lu_env *env,cfs_hash_bd_get_and_lock (hs, (void *)f, &bd,1 ); o =htable_lookup (s, &bd, f, waiter, &version);cfs_hash_bd_unlock (hs, &bd,1 ); -if (o !=NULL ) +if (!IS_ERR (o) ||PTR_ERR (o) != -ENOENT )return o;/*
@@ -667,7 +667,7 @@ static struct lu_object *lu_object_find_try(const struct lu_env *env,cfs_hash_bd_lock (hs, &bd,1 ); shadow =htable_lookup (s, &bd, f, waiter, &version); -if (likely (shadow ==NULL )) { +if (likely (IS_ERR (shadow) &&PTR_ERR (shadow) == -ENOENT )) {struct lu_site_bkt_data *bkt; bkt =cfs_hash_bd_extra_get (hs, &bd);
@@ -2077,7 +2077,7 @@ void lu_object_assign_fid(const struct lu_env *env, struct lu_object *o,cfs_hash_bd_get_and_lock (hs, (void *)fid, &bd,1 ); shadow =htable_lookup (s, &bd, fid, &waiter, &version);/* supposed to be unique */ -LASSERT (shadow ==NULL ); +LASSERT (IS_ERR (shadow) &&PTR_ERR (shadow) == -ENOENT ); *old = *fid; bkt =cfs_hash_bd_extra_get (hs, &bd);cfs_hash_bd_add_locked (hs, &bd, &o->lo_header->loh_hash);
Signature · 1 line
1.7.9.5
PATCH 48/48 staging/lustre/mdc: Keep resend FLocks
In reply to Peng TaoFrom: Bruno Faccini <[email protected]>
FLocks requests (particulary F_UNLCKs) can't be trashed upon comm problems with Server/MDS nor upon kill/exit, thus we need to keep retry/send.
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2665
Lustre-change: http://review.whamcloud.com/6415
--- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/staging/lustre/lustre/mdc/mdc_locks.c +7 -1
@@ -823,7 +823,13 @@ resend:/* For flock requests we immediatelly return without further delay and let caller deal with the rest, since rest of this function metadata processing makes no sense for flock -requests anyway */ +requests anyway. But in case of problem during comms with +Server (ETIMEDOUT) or any signal/kill attempt (EINTR), we +can not rely on caller and this mainly for F_UNLCKs +(explicits or automatically generated by Kernel to clean +current FLocks upon exit) that can't be trashed */ +if ((rc == -EINTR ) || (rc == -ETIMEDOUT )) + goto resend;RETURN (rc); }
Signature · 1 line
1.7.9.5
Greg Kroah-Hartman
maintainerQuoted 5 lines · Peng Tao
Hi Greg,
The first two of the patchset are minor cleanup of lustre code. And the rest of it are patches ported from Intel Lustre tree. Please help to queue them.
I've applied most of these, see my comments on the ones I didn't.
Feel free to fix those up and resend.
Also please resync on my tree, as I tool some other lustre patches from different developers that might conflict with your future changes.
thanks,
greg k-h
Quoted 7 lines · Greg Kroah-Hartman
Quoted 5 lines · Peng Tao
Hi Greg,
The first two of the patchset are minor cleanup of lustre code. And the rest of it are patches ported from Intel Lustre tree. Please help to queue them.
Quoted 7 lines
I've applied most of these, see my comments on the ones I didn't.
Feel free to fix those up and resend.
Also please resync on my tree, as I tool some other lustre patches from different developers that might conflict with your future changes.
Thanks. I will fix up, rebase and resend.
Thanks,
Tao