SUNRPC: Replace dprintk calls with tracepoints
Subsystems (4)
cover SUNRPC: Replace dprintk calls with tracepoints
Hi-
This series replaces many client-side RPC dprintk call sites with tracepoints. The goals of this series are:
- Replace chatty dprintk call sites with tracepoints, which can handle a higher event rate, and won't get rate-limited.
- At some later point, expand the 0-64K range of RPC task IDs. Task IDs would be displayed only by tracepoints as 32-bit unsigned integers.
- Eliminate redundant tracepoints in the transport implementations.
---
Chuck Lever (22):
SUNRPC: Remove trace_xprt_complete_rqst()
SUNRPC: Hoist trace_xprtrdma_op_allocate into generic code
SUNRPC: Remove debugging instrumentation from xprt_release
SUNRPC: Update debugging instrumentation in xprt_do_reserve()
SUNRPC: Replace dprintk() call site in xprt_prepare_transmit
SUNRPC: Replace dprintk() call site in xs_nospace()
SUNRPC: Remove the dprint_status() macro
SUNRPC: Remove dprintk call site in call_start()
SUNRPC: Replace connect dprintk call sites with a tracepoint
SUNRPC: Mitigate cond_resched() in xprt_transmit()
SUNRPC: Add trace_rpc_timeout_status()
SUNRPC: Trace call_refresh events
SUNRPC: Remove dprintk call site in call_decode
SUNRPC: Clean up call_bind_status() observability
SUNRPC: Remove rpcb_getport_async dprintk call sites
SUNRPC: Hoist trace_xprtrdma_op_setport into generic code
SUNRPC: Remove dprintk call sites in rpcbind XDR functions
SUNRPC: Remove more dprintks in rpcb_clnt.c
SUNRPC: Replace rpcbind dprintk call sites with tracepoints
SUNRPC: Clean up RPC scheduler tracepoints
SUNRPC: Remove dprintk call sites in RPC queuing functions
SUNRPC: Remove remaining dprintks from sched.c
include/trace/events/rpcrdma.h | 63 ------- include/trace/events/sunrpc.h | 285 ++++++++++++++++++++++++++++---- net/sunrpc/clnt.c | 75 ++------- net/sunrpc/rpcb_clnt.c | 129 +++------------ net/sunrpc/sched.c | 52 +----- net/sunrpc/xprt.c | 22 +-- net/sunrpc/xprtrdma/transport.c | 7 - net/sunrpc/xprtsock.c | 5 +- 8 files changed, 304 insertions(+), 334 deletions(-)
--
Chuck Lever
PATCH 1/22 SUNRPC: Remove trace_xprt_complete_rqst()
Request completion is already recorded by an "rpc_task_wakeup queue=xprt_pending" trace record. A subsequent rpc_xdr_recvfrom trace record shows the number of bytes received.
--- include/trace/events/sunrpc.h | 1 - net/sunrpc/xprt.c | 2 -- 2 files changed, 3 deletions(-)
include/trace/events/sunrpc.h +0 -1
@@ -969,7 +969,6 @@ DECLARE_EVENT_CLASS(rpc_xprt_event,DEFINE_RPC_XPRT_EVENT (timer);DEFINE_RPC_XPRT_EVENT (lookup_rqst); -DEFINE_RPC_XPRT_EVENT (complete_rqst); TRACE_EVENT(xprt_transmit, TP_PROTO(
net/sunrpc/xprt.c +0 -2
@@ -1122,8 +1122,6 @@ void xprt_complete_rqst(struct rpc_task *task, int copied)struct rpc_rqst *req = task->tk_rqstp;struct rpc_xprt *xprt = req->rq_xprt; -trace_xprt_complete_rqst (xprt, req->rq_xid, copied); - xprt->stat.recvs++; req->rq_private_buf.len = copied;
PATCH 2/22 SUNRPC: Hoist trace_xprtrdma_op_allocate into generic code
In reply to Chuck LeverIntroduce a tracepoint in call_allocate that reports the exact sizes in the RPC buffer allocation request and the status of the result. This helps catch problems with XDR buffer provisioning, and replaces transport-specific debugging instrumentation.
--- include/trace/events/rpcrdma.h | 30 ------------------------------ include/trace/events/sunrpc.h | 30 ++++++++++++++++++++++++++++++ net/sunrpc/clnt.c | 3 +-- net/sunrpc/sched.c | 2 -- net/sunrpc/xprtrdma/transport.c | 2 -- 5 files changed, 31 insertions(+), 36 deletions(-)
include/trace/events/rpcrdma.h +0 -30
@@ -1150,36 +1150,6 @@ TRACE_EVENT(xprtrdma_decode_seg,** Allocation/release of rpcrdma_reqs and rpcrdma_reps **/ -TRACE_EVENT (xprtrdma_op_allocate, -TP_PROTO ( - const struct rpc_task *task, - const struct rpcrdma_req *req - ), - -TP_ARGS (task, req), - -TP_STRUCT__entry ( - __field(unsigned int, task_id) - __field(unsigned int, client_id) - __field(const void *, req) - __field(size_t, callsize) -__field (size_t, rcvsize) - ), - -TP_fast_assign ( - __entry->task_id = task->tk_pid; - __entry->client_id = task->tk_client->cl_clid; - __entry->req = req; - __entry->callsize = task->tk_rqstp->rq_callsize; - __entry->rcvsize = task->tk_rqstp->rq_rcvsize; - ), - -TP_printk ("task:%u@%u req=%p (%zu, %zu)" , - __entry->task_id, __entry->client_id, - __entry->req, __entry->callsize, __entry->rcvsize - ) -); - TRACE_EVENT(xprtrdma_op_free, TP_PROTO( const struct rpc_task *task,
include/trace/events/sunrpc.h +30 -0
@@ -517,6 +517,36 @@ DEFINE_RPC_REPLY_EVENT(stale_creds);DEFINE_RPC_REPLY_EVENT (bad_creds);DEFINE_RPC_REPLY_EVENT (auth_tooweak); +TRACE_EVENT (rpc_buf_alloc, +TP_PROTO ( + const struct rpc_task *task, + int status + ), + +TP_ARGS (task, status), + +TP_STRUCT__entry ( + __field(unsigned int, task_id) + __field(unsigned int, client_id) + __field(size_t, callsize) + __field(size_t, recvsize) +__field (int, status) + ), + +TP_fast_assign ( + __entry->task_id = task->tk_pid; + __entry->client_id = task->tk_client->cl_clid; + __entry->callsize = task->tk_rqstp->rq_callsize; + __entry->recvsize = task->tk_rqstp->rq_rcvsize; + __entry->status = status; + ), + +TP_printk ("task:%u@%u callsize=%zu recvsize=%zu status=%d" , + __entry->task_id, __entry->client_id, + __entry->callsize, __entry->recvsize, __entry->status + ) +); + TRACE_EVENT(rpc_call_rpcerror, TP_PROTO( const struct rpc_task *task,
net/sunrpc/clnt.c +1 -2
@@ -1823,6 +1823,7 @@ call_allocate(struct rpc_task *task) req->rq_rcvsize <<=2 ; status = xprt->ops->buf_alloc (task); +trace_rpc_buf_alloc (task, status);xprt_inject_disconnect (xprt);if (status ==0 )return ;
@@ -1831,8 +1832,6 @@ call_allocate(struct rpc_task *task)return ; } -dprintk ("RPC: %5u rpc_buffer allocation failed\n" , task->tk_pid); -if (RPC_IS_ASYNC (task) || !fatal_signal_pending (current)) { task->tk_action = call_allocate;rpc_delay (task,HZ >>4 );
net/sunrpc/sched.c +0 -2
@@ -1036,8 +1036,6 @@ int rpc_malloc(struct rpc_task *task)return -ENOMEM ; buf->len = size; -dprintk ("RPC: %5u allocated buffer of size %zu at %p\n" , - task->tk_pid, size, buf); rqst->rq_buffer = buf->data; rqst->rq_rbuffer = (char *)rqst->rq_buffer + rqst->rq_callsize;return 0 ;
net/sunrpc/xprtrdma/transport.c +0 -2
@@ -586,11 +586,9 @@ xprt_rdma_allocate(struct rpc_task *task) rqst->rq_buffer =rdmab_data (req->rl_sendbuf); rqst->rq_rbuffer =rdmab_data (req->rl_recvbuf); -trace_xprtrdma_op_allocate (task, req);return 0 ; out_fail: -trace_xprtrdma_op_allocate (task,NULL );return -ENOMEM ; }
PATCH 3/22 SUNRPC: Remove debugging instrumentation from xprt_release
In reply to Chuck LeverThese instruments don't appear to add any substantial value.
We already have this at the termination of each RPC:
iozone-2617 [002] 975.713126: rpc_stats_latency: task:418@5 xid=0x260eab5d nfsv3 LOOKUP backlog=15 rtt=32 execute=58
iozone-2617 [002] 975.713127: xprt_release_cong: task:418@5 snd_task:4294967295 cong=256 cwnd=16384
iozone-2617 [002] 975.713127: xprt_put_cong: task:418@5 snd_task:4294967295 cong=0 cwnd=16384--- include/trace/events/rpcrdma.h | 32 -------------------------------- net/sunrpc/sched.c | 3 --- net/sunrpc/xprt.c | 1 - net/sunrpc/xprtrdma/transport.c | 2 -- 4 files changed, 38 deletions(-)
include/trace/events/rpcrdma.h +0 -32
@@ -1146,38 +1146,6 @@ TRACE_EVENT(xprtrdma_decode_seg, ) ); -/** -** Allocation/release of rpcrdma_reqs and rpcrdma_reps -**/ - -TRACE_EVENT (xprtrdma_op_free, -TP_PROTO ( - const struct rpc_task *task, - const struct rpcrdma_req *req - ), - -TP_ARGS (task, req), - -TP_STRUCT__entry ( - __field(unsigned int, task_id) - __field(unsigned int, client_id) - __field(const void *, req) - __field(const void *, rep) - ), - -TP_fast_assign ( - __entry->task_id = task->tk_pid; - __entry->client_id = task->tk_client->cl_clid; - __entry->req = req; - __entry->rep = req->rl_reply; - ), - -TP_printk ("task:%u@%u req=%p rep=%p" , - __entry->task_id, __entry->client_id, - __entry->req, __entry->rep - ) -); -/** ** Callback events **/
net/sunrpc/sched.c +0 -3
@@ -1056,9 +1056,6 @@ void rpc_free(struct rpc_task *task) buf = container_of(buffer, struct rpc_buffer, data); size = buf->len; -dprintk ("RPC: freeing buffer of size %zu at %p\n" , - size, buf); -if (size <=RPC_BUFFER_MAXSIZE )mempool_free (buf, rpc_buffer_mempool);else
net/sunrpc/xprt.c +0 -1
@@ -1827,7 +1827,6 @@ void xprt_release(struct rpc_task *task)if (req->rq_release_snd_buf) req->rq_release_snd_buf (req); -dprintk ("RPC: %5u release request %p\n" , task->tk_pid, req);if (likely (!bc_prealloc (req))) xprt->ops->free_slot (xprt, req);else
net/sunrpc/xprtrdma/transport.c +0 -2
@@ -605,8 +605,6 @@ xprt_rdma_free(struct rpc_task *task)struct rpcrdma_xprt *r_xprt =rpcx_to_rdmax (rqst->rq_xprt);struct rpcrdma_req *req =rpcr_to_rdmar (rqst); -trace_xprtrdma_op_free (task, req); -if (!list_empty (&req->rl_registered))frwr_unmap_sync (r_xprt, req);
PATCH 4/22 SUNRPC: Update debugging instrumentation in xprt_do_reserve()
In reply to Chuck LeverReplace a dprintk() with a tracepoint. The tracepoint marks the point where an RPC request is assigned an XID.
Additional clean up: Remove trace_xprt_enq_xmit, which reports much the same thing. That tracepoint was added for debugging commit 918f3c1fe83c ("SUNRPC: Improve latency for interactive tasks").
--- include/trace/events/sunrpc.h | 55 ++++++++++++++++++----------------------- net/sunrpc/xprt.c | 8 +----- 2 files changed, 26 insertions(+), 37 deletions(-)
include/trace/events/sunrpc.h +24 -31
@@ -1031,37 +1031,6 @@ TRACE_EVENT(xprt_transmit, __entry->seqno, __entry->status) ); -TRACE_EVENT (xprt_enq_xmit, -TP_PROTO ( - const struct rpc_task *task, - int stage - ), - -TP_ARGS (task, stage), - -TP_STRUCT__entry ( - __field(unsigned int, task_id) - __field(unsigned int, client_id) - __field(u32, xid) - __field(u32, seqno) -__field (int, stage) - ), - -TP_fast_assign ( - __entry->task_id = task->tk_pid; - __entry->client_id = task->tk_client ? - task->tk_client->cl_clid : -1; - __entry->xid = be32_to_cpu(task->tk_rqstp->rq_xid); - __entry->seqno = task->tk_rqstp->rq_seqno; - __entry->stage = stage; - ), - -TP_printk ( -"task:%u@%u xid=0x%08x seqno=%u stage=%d" , - __entry->task_id, __entry->client_id, __entry->xid, - __entry->seqno, __entry->stage) -); - TRACE_EVENT(xprt_ping, TP_PROTO(const struct rpc_xprt *xprt, int status),
@@ -1176,6 +1145,30 @@ DEFINE_CONG_EVENT(release_cong);DEFINE_CONG_EVENT (get_cong);DEFINE_CONG_EVENT (put_cong); +TRACE_EVENT (xprt_reserve, +TP_PROTO ( + const struct rpc_rqst *rqst + ), + +TP_ARGS (rqst), + +TP_STRUCT__entry ( + __field(unsigned int, task_id) + __field(unsigned int, client_id) +__field (u32, xid) + ), + +TP_fast_assign ( + __entry->task_id = rqst->rq_task->tk_pid; + __entry->client_id = rqst->rq_task->tk_client->cl_clid; + __entry->xid = be32_to_cpu(rqst->rq_xid); + ), + +TP_printk ("task:%u@%u xid=0x%08x" , + __entry->task_id, __entry->client_id, __entry->xid + ) +); + TRACE_EVENT(xs_stream_read_data, TP_PROTO(struct rpc_xprt *xprt, ssize_t err, size_t total),
net/sunrpc/xprt.c +2 -6
@@ -1258,7 +1258,6 @@ xprt_request_enqueue_transmit(struct rpc_task *task)/* Note: req is added _before_ pos */ list_add_tail (&req->rq_xmit, &pos->rq_xmit);INIT_LIST_HEAD (&req->rq_xmit2); -trace_xprt_enq_xmit (task,1 ); goto out; } }else if (RPC_IS_SWAPPER (task)) {
@@ -1270,7 +1269,6 @@ xprt_request_enqueue_transmit(struct rpc_task *task)/* Note: req is added _before_ pos */ list_add_tail (&req->rq_xmit, &pos->rq_xmit);INIT_LIST_HEAD (&req->rq_xmit2); -trace_xprt_enq_xmit (task,2 ); goto out; } }else if (!req->rq_seqno) {
@@ -1279,13 +1277,11 @@ xprt_request_enqueue_transmit(struct rpc_task *task)continue ;list_add_tail (&req->rq_xmit2, &pos->rq_xmit2);INIT_LIST_HEAD (&req->rq_xmit); -trace_xprt_enq_xmit (task,3 ); goto out; } }list_add_tail (&req->rq_xmit, &xprt->xmit_queue);INIT_LIST_HEAD (&req->rq_xmit2); -trace_xprt_enq_xmit (task,4 ); out:set_bit (RPC_TASK_NEED_XMIT , &task->tk_runstate);spin_unlock (&xprt->queue_lock);
@@ -1736,8 +1732,8 @@ xprt_request_init(struct rpc_task *task) req->rq_rcv_buf.bvec =NULL ; req->rq_release_snd_buf =NULL ;xprt_init_majortimeo (task, req); -dprintk ("RPC: %5u reserved req %p xid %08x\n" , task->tk_pid, - req,ntohl (req->rq_xid)); + +trace_xprt_reserve (req); } static void
PATCH 5/22 SUNRPC: Replace dprintk() call site in xprt_prepare_transmit
In reply to Chuck LeverGenerate a trace event when an RPC request is queued without being sent immediately.
--- include/trace/events/sunrpc.h | 1 + net/sunrpc/xprt.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-)
include/trace/events/sunrpc.h +1 -0
@@ -1093,6 +1093,7 @@ DECLARE_EVENT_CLASS(xprt_writelock_event,DEFINE_WRITELOCK_EVENT (reserve_xprt);DEFINE_WRITELOCK_EVENT (release_xprt); +DEFINE_WRITELOCK_EVENT (transmit_queued); DECLARE_EVENT_CLASS(xprt_cong_event, TP_PROTO(
net/sunrpc/xprt.c +2 -2
@@ -1399,9 +1399,9 @@ bool xprt_prepare_transmit(struct rpc_task *task)struct rpc_rqst *req = task->tk_rqstp;struct rpc_xprt *xprt = req->rq_xprt; -dprintk ("RPC: %5u xprt_prepare_transmit\n" , task->tk_pid); -if (!xprt_lock_write (xprt, task)) { +trace_xprt_transmit_queued (xprt, task); +/* Race breaker: someone may have transmitted us */ if (!test_bit (RPC_TASK_NEED_XMIT , &task->tk_runstate)) rpc_wake_up_queued_task_set_status(&xprt->sending,
PATCH 6/22 SUNRPC: Replace dprintk() call site in xs_nospace()
In reply to Chuck Lever"no socket space" is an exceptional and infrequent condition that troubleshooters want to know about.
--- include/trace/events/sunrpc.h | 28 ++++++++++++++++++++++++++++ net/sunrpc/xprtsock.c | 5 +---- 2 files changed, 29 insertions(+), 4 deletions(-)
include/trace/events/sunrpc.h +28 -0
@@ -898,6 +898,34 @@ DEFINE_RPC_SOCKET_EVENT_DONE(rpc_socket_reset_connection);DEFINE_RPC_SOCKET_EVENT (rpc_socket_close);DEFINE_RPC_SOCKET_EVENT (rpc_socket_shutdown); +TRACE_EVENT (rpc_socket_nospace, +TP_PROTO ( + const struct rpc_rqst *rqst, + const struct sock_xprt *transport + ), + +TP_ARGS (rqst, transport), + +TP_STRUCT__entry ( + __field(unsigned int, task_id) + __field(unsigned int, client_id) + __field(unsigned int, total) + __field(unsigned int, remaining) + ), + +TP_fast_assign ( + __entry->task_id = rqst->rq_task->tk_pid; + __entry->client_id = rqst->rq_task->tk_client->cl_clid; + __entry->total = rqst->rq_slen; + __entry->remaining = rqst->rq_slen - transport->xmit.offset; + ), + +TP_printk ("task:%u@%u total=%u remaining=%u" , + __entry->task_id, __entry->client_id, + __entry->total, __entry->remaining + ) +); +TRACE_DEFINE_ENUM (XPRT_LOCKED );TRACE_DEFINE_ENUM (XPRT_CONNECTED );TRACE_DEFINE_ENUM (XPRT_CONNECTING );
net/sunrpc/xprtsock.c +1 -4
@@ -762,10 +762,7 @@ static int xs_nospace(struct rpc_rqst *req)struct sock *sk = transport->inet;int ret = -EAGAIN ; -dprintk ("RPC: %5u xmit incomplete (%u left of %u)\n" , - req->rq_task->tk_pid, - req->rq_slen - transport->xmit.offset, - req->rq_slen); +trace_rpc_socket_nospace (req, transport);/* Protect against races with write_space */ spin_lock (&xprt->transport_lock);
PATCH 7/22 SUNRPC: Remove the dprint_status() macro
In reply to Chuck LeverClean up: The rpc_task_run_action tracepoint serves the same purpose.
--- net/sunrpc/clnt.c | 31 +------------------------------ 1 file changed, 1 insertion(+), 30 deletions(-)
net/sunrpc/clnt.c +1 -30
@@ -47,10 +47,6 @@# define RPCDBG_FACILITY RPCDBG_CALL#endif -#define dprint_status (t) \ - dprintk("RPC: %5u %s (status %d)\n", t->tk_pid, \ - __func__, t->tk_status) -/* * All RPC clients are linked into this list */
@@ -1658,8 +1654,6 @@ call_start(struct rpc_task *task)static void call_reserve (struct rpc_task *task) { -dprint_status (task); - task->tk_status =0 ; task->tk_action = call_reserveresult;xprt_reserve (task);
@@ -1675,8 +1669,6 @@ call_reserveresult(struct rpc_task *task) {int status = task->tk_status; -dprint_status (task); -/* * After a call to xprt_reserve(), we must have either * a request slot or else an error status.
@@ -1717,8 +1709,6 @@ call_reserveresult(struct rpc_task *task)static void call_retry_reserve (struct rpc_task *task) { -dprint_status (task); - task->tk_status =0 ; task->tk_action = call_reserveresult;xprt_retry_reserve (task);
@@ -1730,8 +1720,6 @@ call_retry_reserve(struct rpc_task *task)static void call_refresh (struct rpc_task *task) { -dprint_status (task); - task->tk_action = call_refreshresult; task->tk_status =0 ; task->tk_client->cl_stats->rpcauthrefresh++;
@@ -1746,8 +1734,6 @@ call_refreshresult(struct rpc_task *task) {int status = task->tk_status; -dprint_status (task); - task->tk_status =0 ; task->tk_action = call_refresh;switch (status) {
@@ -1792,8 +1778,6 @@ call_allocate(struct rpc_task *task)const struct rpc_procinfo *proc = task->tk_msg.rpc_proc;int status; -dprint_status (task); - task->tk_status =0 ; task->tk_action = call_encode;
@@ -1882,7 +1866,7 @@ call_encode(struct rpc_task *task) {if (!rpc_task_need_encode (task)) goto out; -dprint_status (task); +/* Dequeue task from the receive queue while we're encoding */ xprt_request_dequeue_xprt (task);/* Encode here so that rpcsec_gss can use correct sequence number. */
@@ -1959,8 +1943,6 @@ call_bind(struct rpc_task *task)return ; } -dprint_status (task); - task->tk_action = call_bind_status;if (!xprt_prepare_transmit (task))return ;
@@ -1982,7 +1964,6 @@ call_bind_status(struct rpc_task *task)return ; } -dprint_status (task);trace_rpc_bind_status (task);if (task->tk_status >=0 ) goto out_next;
@@ -2109,7 +2090,6 @@ call_connect_status(struct rpc_task *task)return ; } -dprint_status (task);trace_rpc_connect_status (task);if (task->tk_status ==0 ) {
@@ -2177,8 +2157,6 @@ call_transmit(struct rpc_task *task)return ; } -dprint_status (task); - task->tk_action = call_transmit_status;if (!xprt_prepare_transmit (task))return ;
@@ -2213,7 +2191,6 @@ call_transmit_status(struct rpc_task *task)switch (task->tk_status) {default : -dprint_status (task);break ;case -EBADMSG : task->tk_status =0 ;
@@ -2295,8 +2272,6 @@ call_bc_transmit_status(struct rpc_task *task)if (rpc_task_transmitted (task)) task->tk_status =0 ; -dprint_status (task); -switch (task->tk_status) {case 0 :/* Success */
@@ -2356,8 +2331,6 @@ call_status(struct rpc_task *task)if (!task->tk_msg.rpc_proc->p_proc)trace_xprt_ping (task->tk_xprt, task->tk_status); -dprint_status (task); - status = task->tk_status;if (status >=0 ) { task->tk_action = call_decode;
@@ -2491,8 +2464,6 @@ call_decode(struct rpc_task *task)struct xdr_stream xdr;int err; -dprint_status (task); -if (!task->tk_msg.rpc_proc->p_decode) { task->tk_action = rpc_exit_task;return ;
PATCH 8/22 SUNRPC: Remove dprintk call site in call_start()
In reply to Chuck LeverClean up: The rpc_rpc_request tracepoint serves the same purpose.
--- net/sunrpc/clnt.c | 4 ---- 1 file changed, 4 deletions(-)
net/sunrpc/clnt.c +0 -4
@@ -1635,10 +1635,6 @@ call_start(struct rpc_task *task)int idx = task->tk_msg.rpc_proc->p_statidx;trace_rpc_request (task); -dprintk ("RPC: %5u call_start %s%d proc %s (%s)\n" , task->tk_pid, - clnt->cl_program->name, clnt->cl_vers, -rpc_proc_name (task), - (RPC_IS_ASYNC (task) ?"async" :"sync" ));/* Increment call count (version might not be valid for ping) */ if (clnt->cl_program->version[clnt->cl_vers])
PATCH 9/22 SUNRPC: Replace connect dprintk call sites with a tracepoint
In reply to Chuck LeverThis trace event can be used to audit transport connections from the client.
--- include/trace/events/sunrpc.h | 1 + net/sunrpc/clnt.c | 4 ---- net/sunrpc/xprt.c | 3 +-- 3 files changed, 2 insertions(+), 6 deletions(-)
include/trace/events/sunrpc.h +1 -0
@@ -983,6 +983,7 @@ DECLARE_EVENT_CLASS(rpc_xprt_lifetime_class, TP_ARGS(xprt))DEFINE_RPC_XPRT_LIFETIME_EVENT (create); +DEFINE_RPC_XPRT_LIFETIME_EVENT (connect);DEFINE_RPC_XPRT_LIFETIME_EVENT (disconnect_auto);DEFINE_RPC_XPRT_LIFETIME_EVENT (disconnect_done);DEFINE_RPC_XPRT_LIFETIME_EVENT (disconnect_force);
net/sunrpc/clnt.c +0 -4
@@ -2055,10 +2055,6 @@ call_connect(struct rpc_task *task)return ; } -dprintk ("RPC: %5u call_connect xprt %p %s connected\n" , - task->tk_pid, xprt, - (xprt_connected (xprt) ?"is" :"is not" )); - task->tk_action = call_connect_status;if (task->tk_status <0 )return ;
net/sunrpc/xprt.c +1 -2
@@ -825,8 +825,7 @@ void xprt_connect(struct rpc_task *task) {struct rpc_xprt *xprt = task->tk_rqstp->rq_xprt; -dprintk ("RPC: %5u xprt_connect xprt %p %s connected\n" , task->tk_pid, - xprt, (xprt_connected (xprt) ?"is" :"is not" )); +trace_xprt_connect (xprt);if (!xprt_bound (xprt)) { task->tk_status = -EAGAIN ;
PATCH 10/22 SUNRPC: Mitigate cond_resched() in xprt_transmit()
In reply to Chuck LeverThe original purpose of this expensive call is to prevent a long queue of requests from blocking other work.
The cond_resched() call is unnecessary after just a single send operation.
For longer queues, instead of invoking the kernel scheduler, simply release the transport send lock and return to the RPC scheduler.
--- net/sunrpc/xprt.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
net/sunrpc/xprt.c +4 -2
@@ -1504,10 +1504,13 @@ xprt_transmit(struct rpc_task *task) {struct rpc_rqst *next, *req = task->tk_rqstp;struct rpc_xprt *xprt = req->rq_xprt; -int status; +int counter, status;spin_lock (&xprt->queue_lock); + counter =0 ;while (!list_empty (&xprt->xmit_queue)) { +if (++counter ==20 ) +break ; next =list_first_entry (&xprt->xmit_queue, struct rpc_rqst, rq_xmit);xprt_pin_rqst (next);
@@ -1515,7 +1518,6 @@ xprt_transmit(struct rpc_task *task) status =xprt_request_transmit (next, task);if (status == -EBADMSG && next != req) status =0 ; -cond_resched ();spin_lock (&xprt->queue_lock);xprt_unpin_rqst (next);if (status ==0 ) {
PATCH 11/22 SUNRPC: Add trace_rpc_timeout_status()
In reply to Chuck LeverFor a long while we've wanted a tracepoint that fires when a major timeout is reported in the system log. Such a tracepoint can be attached to other actions that can take place when a timeout is detected (eg, server or connection health assessment).
--- include/trace/events/sunrpc.h | 1 + net/sunrpc/clnt.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
include/trace/events/sunrpc.h +1 -0
@@ -261,6 +261,7 @@ DECLARE_EVENT_CLASS(rpc_task_status,DEFINE_RPC_STATUS_EVENT (call);DEFINE_RPC_STATUS_EVENT (bind);DEFINE_RPC_STATUS_EVENT (connect); +DEFINE_RPC_STATUS_EVENT (timeout); TRACE_EVENT(rpc_request, TP_PROTO(const struct rpc_task *task),
net/sunrpc/clnt.c +1 -1
@@ -2397,7 +2397,7 @@ rpc_check_timeout(struct rpc_task *task)if (xprt_adjust_timeout (task->tk_rqstp) ==0 )return ; -dprintk ("RPC: %5u call_timeout (major)\n" , task->tk_pid); +trace_rpc_timeout_status (task); task->tk_timeouts++;if (RPC_IS_SOFTCONN (task) && !rpc_check_connected (task->tk_rqstp)) {
PATCH 12/22 SUNRPC: Trace call_refresh events
In reply to Chuck LeverClean up: Replace dprintk call sites.
Note that rpc_call_rpcerror() already has a trace point, so perhaps adding trace_rpc_refresh_status() isn't necessary. However, it does report a particular category of error.
--- include/trace/events/sunrpc.h | 2 ++ net/sunrpc/clnt.c | 9 +++------ 2 files changed, 5 insertions(+), 6 deletions(-)
include/trace/events/sunrpc.h +2 -0
@@ -262,6 +262,8 @@ DEFINE_RPC_STATUS_EVENT(call);DEFINE_RPC_STATUS_EVENT (bind);DEFINE_RPC_STATUS_EVENT (connect);DEFINE_RPC_STATUS_EVENT (timeout); +DEFINE_RPC_STATUS_EVENT (retry_refresh); +DEFINE_RPC_STATUS_EVENT (refresh); TRACE_EVENT(rpc_request, TP_PROTO(const struct rpc_task *task),
net/sunrpc/clnt.c +3 -6
@@ -1752,12 +1752,10 @@ call_refreshresult(struct rpc_task *task)if (!task->tk_cred_retry)break ; task->tk_cred_retry--; -dprintk ("RPC: %5u %s: retry refresh creds\n" , - task->tk_pid, __func__); +trace_rpc_retry_refresh_status (task);return ; } -dprintk ("RPC: %5u %s: refresh creds failed with error %d\n" , - task->tk_pid, __func__, status); +trace_rpc_refresh_status (task);rpc_call_rpcerror (task, status); }
@@ -1881,8 +1879,7 @@ call_encode(struct rpc_task *task) }else { task->tk_action = call_refresh; task->tk_cred_retry--; -dprintk ("RPC: %5u %s: retry refresh creds\n" , - task->tk_pid, __func__); +trace_rpc_retry_refresh_status (task); }break ;default :
PATCH 13/22 SUNRPC: Remove dprintk call site in call_decode
In reply to Chuck LeverClean up.
When enabled, this dprintk adds a line in /var/log/messages after every RPC that reports the task ID (no connection to on the wire XID values) and the RPC's result (no connection to the program, operation, or the arguments and results).
Thus it's value is pretty low. Let's remove it.
--- net/sunrpc/clnt.c | 2 -- 1 file changed, 2 deletions(-)
net/sunrpc/clnt.c +0 -2
@@ -2496,8 +2496,6 @@ call_decode(struct rpc_task *task)case 0 : task->tk_action = rpc_exit_task; task->tk_status =rpcauth_unwrap_resp (task, &xdr); -dprintk ("RPC: %5u %s result %d\n" , - task->tk_pid, __func__, task->tk_status);return ;case -EAGAIN : task->tk_status =0 ;
PATCH 14/22 SUNRPC: Clean up call_bind_status() observability
In reply to Chuck LeverTime to remove dprintk call sites in here.
Regarding the rpc_bind_status tracepoint: It's friendlier to administrators if they don't have to look up the error code to figure out what went wrong. Replace trace_rpc_bind_status with a set of tracepoints that report more specifically what the problem was, and what RPC program/version was being queried.
--- include/trace/events/sunrpc.h | 14 +++++++++++++- net/sunrpc/clnt.c | 20 ++++++-------------- 2 files changed, 19 insertions(+), 15 deletions(-)
include/trace/events/sunrpc.h +13 -1
@@ -259,7 +259,6 @@ DECLARE_EVENT_CLASS(rpc_task_status, TP_ARGS(task))DEFINE_RPC_STATUS_EVENT (call); -DEFINE_RPC_STATUS_EVENT (bind);DEFINE_RPC_STATUS_EVENT (connect);DEFINE_RPC_STATUS_EVENT (timeout);DEFINE_RPC_STATUS_EVENT (retry_refresh);
@@ -520,6 +519,19 @@ DEFINE_RPC_REPLY_EVENT(stale_creds);DEFINE_RPC_REPLY_EVENT (bad_creds);DEFINE_RPC_REPLY_EVENT (auth_tooweak); +#define DEFINE_RPCB_ERROR_EVENT (name) \ + DEFINE_EVENT(rpc_reply_event, rpcb_##name##_err, \ + TP_PROTO( \ + const struct rpc_task *task \ + ), \ + TP_ARGS(task)) + +DEFINE_RPCB_ERROR_EVENT (prog_unavail); +DEFINE_RPCB_ERROR_EVENT (timeout); +DEFINE_RPCB_ERROR_EVENT (bind_version); +DEFINE_RPCB_ERROR_EVENT (unreachable); +DEFINE_RPCB_ERROR_EVENT (unrecognized); + TRACE_EVENT(rpc_buf_alloc, TP_PROTO( const struct rpc_task *task,
net/sunrpc/clnt.c +6 -14
@@ -1957,7 +1957,6 @@ call_bind_status(struct rpc_task *task)return ; } -trace_rpc_bind_status (task);if (task->tk_status >=0 ) goto out_next;if (xprt_bound (xprt)) {
@@ -1967,12 +1966,10 @@ call_bind_status(struct rpc_task *task)switch (task->tk_status) {case -ENOMEM : -dprintk ("RPC: %5u rpcbind out of memory\n" , task->tk_pid);rpc_delay (task,HZ >>2 ); goto retry_timeout;case -EACCES : -dprintk ("RPC: %5u remote rpcbind: RPC program/version " -"unavailable\n" , task->tk_pid); +trace_rpcb_prog_unavail_err (task);/* fail immediately if this is an RPC ping */ if (task->tk_msg.rpc_proc->p_proc ==0 ) { status = -EOPNOTSUPP ;
@@ -1989,17 +1986,14 @@ call_bind_status(struct rpc_task *task)case -EAGAIN : goto retry_timeout;case -ETIMEDOUT : -dprintk ("RPC: %5u rpcbind request timed out\n" , - task->tk_pid); +trace_rpcb_timeout_err (task); goto retry_timeout;case -EPFNOSUPPORT :/* server doesn't support any rpcbind version we know of */ -dprintk ("RPC: %5u unrecognized remote rpcbind service\n" , - task->tk_pid); +trace_rpcb_bind_version_err (task);break ;case -EPROTONOSUPPORT : -dprintk ("RPC: %5u remote rpcbind version unavailable, retrying\n" , - task->tk_pid); +trace_rpcb_bind_version_err (task); goto retry_timeout;case -ECONNREFUSED :/* connection problems */ case -ECONNRESET :
@@ -2010,8 +2004,7 @@ call_bind_status(struct rpc_task *task)case -EHOSTUNREACH :case -ENETUNREACH :case -EPIPE : -dprintk ("RPC: %5u remote rpcbind unreachable: %d\n" , - task->tk_pid, task->tk_status); +trace_rpcb_unreachable_err (task);if (!RPC_IS_SOFTCONN (task)) {rpc_delay (task,5 *HZ ); goto retry_timeout;
@@ -2019,8 +2012,7 @@ call_bind_status(struct rpc_task *task) status = task->tk_status;break ;default : -dprintk ("RPC: %5u unrecognized rpcbind error (%d)\n" , - task->tk_pid, -task->tk_status); +trace_rpcb_unrecognized_err (task); }rpc_call_rpcerror (task, status);
PATCH 15/22 SUNRPC: Remove rpcb_getport_async dprintk call sites
In reply to Chuck LeverIn many cases, tracepoints already report these errors. In others, the dprintks were mainly useful when this code was less mature.
--- include/trace/events/sunrpc.h | 35 +++++++++++++++++++++++++++++++++++ net/sunrpc/rpcb_clnt.c | 23 +++-------------------- 2 files changed, 38 insertions(+), 20 deletions(-)
include/trace/events/sunrpc.h +35 -0
@@ -1269,6 +1269,41 @@ TRACE_EVENT(xs_stream_read_request, __entry->copied, __entry->reclen, __entry->offset) ); +TRACE_EVENT (rpcb_getport, +TP_PROTO ( + const struct rpc_clnt *clnt, + const struct rpc_task *task, + unsigned int bind_version + ), + +TP_ARGS (clnt, task, bind_version), + +TP_STRUCT__entry ( + __field(unsigned int, task_id) + __field(unsigned int, client_id) + __field(unsigned int, program) + __field(unsigned int, version) + __field(int, protocol) + __field(unsigned int, bind_version) +__string (servername, task->tk_xprt->servername) + ), + +TP_fast_assign ( + __entry->task_id = task->tk_pid; + __entry->client_id = clnt->cl_clid; + __entry->program = clnt->cl_prog; + __entry->version = clnt->cl_vers; + __entry->protocol = task->tk_xprt->prot; + __entry->bind_version = bind_version; + __assign_str(servername, task->tk_xprt->servername); + ), + +TP_printk ("task:%u@%u server=%s program=%u version=%u protocol=%d bind_version=%u" , + __entry->task_id, __entry->client_id,__get_str (servername), + __entry->program, __entry->version, __entry->protocol, + __entry->bind_version + ) +); DECLARE_EVENT_CLASS(svc_xdr_buf_class, TP_PROTO(
net/sunrpc/rpcb_clnt.c +3 -20
@@ -31,6 +31,8 @@#include <linux/sunrpc/sched.h> #include <linux/sunrpc/xprtsock.h> +#include <trace/events/sunrpc.h> +#include "netns.h" #if IS_ENABLED (CONFIG_SUNRPC_DEBUG )
@@ -693,18 +695,12 @@ void rpcb_getport_async(struct rpc_task *task)rcu_read_unlock (); xprt =xprt_get (task->tk_xprt); -dprintk ("RPC: %5u %s(%s, %u, %u, %d)\n" , - task->tk_pid, __func__, - xprt->servername, clnt->cl_prog, clnt->cl_vers, xprt->prot); -/* Put self on the wait queue to ensure we get notified if * some other task is already attempting to bind the port */ rpc_sleep_on_timeout (&xprt->binding, task,NULL , jiffies + xprt->bind_timeout);if (xprt_test_and_set_binding (xprt)) { -dprintk ("RPC: %5u %s: waiting for another binder\n" , - task->tk_pid, __func__);xprt_put (xprt);return ; }
@@ -712,8 +708,6 @@ void rpcb_getport_async(struct rpc_task *task)/* Someone else may have bound if we slept */ if (xprt_bound (xprt)) { status =0 ; -dprintk ("RPC: %5u %s: already bound\n" , - task->tk_pid, __func__); goto bailout_nofree; }
@@ -732,20 +726,15 @@ void rpcb_getport_async(struct rpc_task *task)break ;default : status = -EAFNOSUPPORT ; -dprintk ("RPC: %5u %s: bad address family\n" , - task->tk_pid, __func__); goto bailout_nofree; }if (proc ==NULL ) { xprt->bind_index =0 ; status = -EPFNOSUPPORT ; -dprintk ("RPC: %5u %s: no more getport versions available\n" , - task->tk_pid, __func__); goto bailout_nofree; } -dprintk ("RPC: %5u %s: trying rpcbind version %u\n" , - task->tk_pid, __func__, bind_version); +trace_rpcb_getport (clnt, task, bind_version); rpcb_clnt = rpcb_create(xprt->xprt_net, clnt->cl_nodename,
@@ -754,16 +743,12 @@ void rpcb_getport_async(struct rpc_task *task) clnt->cl_cred);if (IS_ERR (rpcb_clnt)) { status =PTR_ERR (rpcb_clnt); -dprintk ("RPC: %5u %s: rpcb_create failed, error %ld\n" , - task->tk_pid, __func__,PTR_ERR (rpcb_clnt)); goto bailout_nofree; } map =kzalloc (sizeof (struct rpcbind_args ),GFP_NOFS );if (!map) { status = -ENOMEM ; -dprintk ("RPC: %5u %s: no memory available\n" , - task->tk_pid, __func__); goto bailout_release_client; } map->r_prog = clnt->cl_prog;
@@ -780,8 +765,6 @@ void rpcb_getport_async(struct rpc_task *task) map->r_addr =rpc_sockaddr2uaddr (sap,GFP_NOFS );if (!map->r_addr) { status = -ENOMEM ; -dprintk ("RPC: %5u %s: no memory available\n" , - task->tk_pid, __func__); goto bailout_free_args; } map->r_owner ="" ;
PATCH 16/22 SUNRPC: Hoist trace_xprtrdma_op_setport into generic code
In reply to Chuck Lever--- include/trace/events/rpcrdma.h | 1 - include/trace/events/sunrpc.h | 29 +++++++++++++++++++++++++++++ net/sunrpc/rpcb_clnt.c | 29 ++++++++++++++--------------- net/sunrpc/xprtrdma/transport.c | 3 --- 4 files changed, 43 insertions(+), 19 deletions(-)
include/trace/events/rpcrdma.h +0 -1
@@ -382,7 +382,6 @@ DEFINE_CONN_EVENT(connect);DEFINE_CONN_EVENT (disconnect);DEFINE_RXPRT_EVENT (xprtrdma_op_inject_dsc); -DEFINE_RXPRT_EVENT (xprtrdma_op_setport); TRACE_EVENT(xprtrdma_op_connect, TP_PROTO(
include/trace/events/sunrpc.h +29 -0
@@ -1305,6 +1305,35 @@ TRACE_EVENT(rpcb_getport, ) ); +TRACE_EVENT (rpcb_setport, +TP_PROTO ( + const struct rpc_task *task, + int status, + unsigned short port + ), + +TP_ARGS (task, status, port), + +TP_STRUCT__entry ( + __field(unsigned int, task_id) + __field(unsigned int, client_id) + __field(int, status) + __field(unsigned short, port) + ), + +TP_fast_assign ( + __entry->task_id = task->tk_pid; + __entry->client_id = task->tk_client->cl_clid; + __entry->status = status; + __entry->port = port; + ), + +TP_printk ("task:%u@%u status=%d port=%u" , + __entry->task_id, __entry->client_id, + __entry->status, __entry->port + ) +); + DECLARE_EVENT_CLASS(svc_xdr_buf_class, TP_PROTO( const struct svc_rqst *rqst,
net/sunrpc/rpcb_clnt.c +14 -15
@@ -801,34 +801,33 @@ static void rpcb_getport_done(struct rpc_task *child, void *data) {struct rpcbind_args *map = data;struct rpc_xprt *xprt = map->r_xprt; -int status = child->tk_status; + + map->r_status = child->tk_status;/* Garbage reply: retry with a lesser rpcbind version */ -if (status == -EIO ) - status = -EPROTONOSUPPORT ; +if (map->r_status == -EIO ) + map->r_status = -EPROTONOSUPPORT ;/* rpcbind server doesn't support this rpcbind protocol version */ -if (status == -EPROTONOSUPPORT ) +if (map->r_status == -EPROTONOSUPPORT ) xprt->bind_index++; -if (status <0 ) { +if (map->r_status <0 ) {/* rpcbind server not available on remote host? */ - xprt->ops->set_port (xprt,0 ); + map->r_port =0 ; + }else if (map->r_port ==0 ) {/* Requested RPC service wasn't registered on remote host */ - xprt->ops->set_port (xprt,0 ); - status = -EACCES ; + map->r_status = -EACCES ; }else {/* Succeeded */ - xprt->ops->set_port (xprt, map->r_port); -xprt_set_bound (xprt); - status =0 ; + map->r_status =0 ; } -dprintk ("RPC: %5u rpcb_getport_done(status %d, port %u)\n" , - child->tk_pid, status, map->r_port); - - map->r_status = status; +trace_rpcb_setport (child, map->r_status, map->r_port); + xprt->ops->set_port (xprt, map->r_port); +if (map->r_port) +xprt_set_bound (xprt); }/*
net/sunrpc/xprtrdma/transport.c +0 -3
@@ -413,9 +413,6 @@ xprt_rdma_set_port(struct rpc_xprt *xprt, u16 port)kfree (xprt->address_strings[RPC_DISPLAY_HEX_PORT ]);snprintf (buf,sizeof (buf),"%4hx" , port); xprt->address_strings[RPC_DISPLAY_HEX_PORT ] =kstrdup (buf,GFP_KERNEL ); - -trace_xprtrdma_op_setport (container_of(xprt ,struct rpcrdma_xprt , -rx_xprt )); }/**
PATCH 17/22 SUNRPC: Remove dprintk call sites in rpcbind XDR functions
In reply to Chuck LeverClean up: Other XDR functions no longer have dprintk call sites. These were added during development and can be removed now that the code is mature.
--- net/sunrpc/rpcb_clnt.c | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-)
net/sunrpc/rpcb_clnt.c +1 -27
@@ -840,11 +840,6 @@ static void rpcb_enc_mapping(struct rpc_rqst *req, struct xdr_stream *xdr,const struct rpcbind_args *rpcb = data;__be32 *p; -dprintk ("RPC: %5u encoding PMAP_%s call (%u, %u, %d, %u)\n" , - req->rq_task->tk_pid, - req->rq_task->tk_msg.rpc_proc->p_name, - rpcb->r_prog, rpcb->r_vers, rpcb->r_prot, rpcb->r_port); - p =xdr_reserve_space (xdr, RPCB_mappingargs_sz <<2 ); *p++ =cpu_to_be32 (rpcb->r_prog); *p++ =cpu_to_be32 (rpcb->r_vers);
@@ -866,8 +861,6 @@ static int rpcb_dec_getport(struct rpc_rqst *req, struct xdr_stream *xdr,return -EIO ; port =be32_to_cpup (p); -dprintk ("RPC: %5u PMAP_%s result: %lu\n" , req->rq_task->tk_pid, - req->rq_task->tk_msg.rpc_proc->p_name, port);if (unlikely (port >USHRT_MAX ))return -EIO ;
@@ -888,11 +881,6 @@ static int rpcb_dec_set(struct rpc_rqst *req, struct xdr_stream *xdr, *boolp =0 ;if (*p != xdr_zero) *boolp =1 ; - -dprintk ("RPC: %5u RPCB_%s call %s\n" , - req->rq_task->tk_pid, - req->rq_task->tk_msg.rpc_proc->p_name, - (*boolp ?"succeeded" :"failed" ));return 0 ; }
@@ -917,12 +905,6 @@ static void rpcb_enc_getaddr(struct rpc_rqst *req, struct xdr_stream *xdr,const struct rpcbind_args *rpcb = data;__be32 *p; -dprintk ("RPC: %5u encoding RPCB_%s call (%u, %u, '%s', '%s')\n" , - req->rq_task->tk_pid, - req->rq_task->tk_msg.rpc_proc->p_name, - rpcb->r_prog, rpcb->r_vers, - rpcb->r_netid, rpcb->r_addr); - p =xdr_reserve_space (xdr, (RPCB_program_sz + RPCB_version_sz) <<2 ); *p++ =cpu_to_be32 (rpcb->r_prog); *p =cpu_to_be32 (rpcb->r_vers);
@@ -952,11 +934,8 @@ static int rpcb_dec_getaddr(struct rpc_rqst *req, struct xdr_stream *xdr,* If the returned universal address is a null string, * the requested RPC service was not registered. */ -if (len ==0 ) { -dprintk ("RPC: %5u RPCB reply: program not registered\n" , - req->rq_task->tk_pid); +if (len ==0 )return 0 ; - }if (unlikely (len >RPCBIND_MAXUADDRLEN )) goto out_fail;
@@ -964,8 +943,6 @@ static int rpcb_dec_getaddr(struct rpc_rqst *req, struct xdr_stream *xdr, p =xdr_inline_decode (xdr, len);if (unlikely (p ==NULL )) goto out_fail; -dprintk ("RPC: %5u RPCB_%s reply: %s\n" , req->rq_task->tk_pid, - req->rq_task->tk_msg.rpc_proc->p_name, (char *)p);if (rpc_uaddr2sockaddr (req->rq_xprt->xprt_net, (char *)p, len, sap,sizeof (address)) ==0 )
@@ -975,9 +952,6 @@ static int rpcb_dec_getaddr(struct rpc_rqst *req, struct xdr_stream *xdr,return 0 ; out_fail: -dprintk ("RPC: %5u malformed RPCB_%s reply\n" , - req->rq_task->tk_pid, - req->rq_task->tk_msg.rpc_proc->p_name);return -EIO ; }
PATCH 18/22 SUNRPC: Remove more dprintks in rpcb_clnt.c
In reply to Chuck LeverClean up: These are superfluous now that rpc_create() and friends have tracepoints to report errors.
--- net/sunrpc/rpcb_clnt.c | 25 +++---------------------- 1 file changed, 3 insertions(+), 22 deletions(-)
net/sunrpc/rpcb_clnt.c +3 -22
@@ -218,10 +218,6 @@ static void rpcb_set_local(struct net *net, struct rpc_clnt *clnt, sn->rpcb_is_af_local = is_af_local ?1 :0 ;smp_wmb (); sn->rpcb_users =1 ; -dprintk ("RPC: created new rpcb local clients (rpcb_local_clnt: " -"%p, rpcb_local_clnt4: %p) for net %x%s\n" , - sn->rpcb_local_clnt, sn->rpcb_local_clnt4, - net->ns.inum, (net == &init_net) ?" (init_net)" :"" ); }/*
@@ -263,19 +259,13 @@ static int rpcb_create_local_unix(struct net *net)*/ clnt =rpc_create (&args);if (IS_ERR (clnt)) { -dprintk ("RPC: failed to create AF_LOCAL rpcbind " -"client (errno %ld).\n" ,PTR_ERR (clnt)); result =PTR_ERR (clnt); goto out; } clnt4 =rpc_bind_new_program (clnt, &rpcb_program,RPCBVERS_4 ); -if (IS_ERR (clnt4)) { -dprintk ("RPC: failed to bind second program to " -"rpcbind v4 client (errno %ld).\n" , -PTR_ERR (clnt4)); +if (IS_ERR (clnt4)) clnt4 =NULL ; - }rpcb_set_local (net, clnt, clnt4, true);
@@ -311,8 +301,6 @@ static int rpcb_create_local_net(struct net *net) clnt =rpc_create (&args);if (IS_ERR (clnt)) { -dprintk ("RPC: failed to create local rpcbind " -"client (errno %ld).\n" ,PTR_ERR (clnt)); result =PTR_ERR (clnt); goto out; }
@@ -323,12 +311,8 @@ static int rpcb_create_local_net(struct net *net)* v4 upcalls. */ clnt4 =rpc_bind_new_program (clnt, &rpcb_program,RPCBVERS_4 ); -if (IS_ERR (clnt4)) { -dprintk ("RPC: failed to bind second program to " -"rpcbind v4 client (errno %ld).\n" , -PTR_ERR (clnt4)); +if (IS_ERR (clnt4)) clnt4 =NULL ; - }rpcb_set_local (net, clnt, clnt4, false);
@@ -405,11 +389,8 @@ static int rpcb_register_call(struct sunrpc_net *sn, struct rpc_clnt *clnt, stru msg->rpc_resp = &result; error =rpc_call_sync (clnt, msg, flags); -if (error <0 ) { -dprintk ("RPC: failed to contact local rpcbind " -"server (errno %d).\n" , -error); +if (error <0 )return error; - }if (!result)return -EACCES ;
PATCH 19/22 SUNRPC: Replace rpcbind dprintk call sites with tracepoints
In reply to Chuck LeverIn many cases, tracepoints already report these errors. In others, the dprintks were mainly useful when this code was less mature.
--- include/trace/events/sunrpc.h | 86 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/rpcb_clnt.c | 24 ++--------- 2 files changed, 90 insertions(+), 20 deletions(-)
include/trace/events/sunrpc.h +86 -0
@@ -1334,6 +1334,92 @@ TRACE_EVENT(rpcb_setport, ) ); +TRACE_EVENT (pmap_register, +TP_PROTO ( + u32 program, + u32 version, + int protocol, + unsigned short port + ), + +TP_ARGS (program, version, protocol, port), + +TP_STRUCT__entry ( + __field(unsigned int, program) + __field(unsigned int, version) + __field(int, protocol) + __field(unsigned int, port) + ), + +TP_fast_assign ( + __entry->program = program; + __entry->version = version; + __entry->protocol = protocol; + __entry->port = port; + ), + +TP_printk ("program=%u version=%u protocol=%d port=%u" , + __entry->program, __entry->version, + __entry->protocol, __entry->port + ) +); + +TRACE_EVENT (rpcb_register, +TP_PROTO ( + u32 program, + u32 version, + const char *addr, + const char *netid + ), + +TP_ARGS (program, version, addr, netid), + +TP_STRUCT__entry ( + __field(unsigned int, program) + __field(unsigned int, version) + __string(addr, addr) +__string (netid, netid) + ), + +TP_fast_assign ( + __entry->program = program; + __entry->version = version; + __assign_str(addr, addr); + __assign_str(netid, netid); + ), + +TP_printk ("program=%u version=%u addr=%s netid=%s" , + __entry->program, __entry->version, +__get_str (addr),__get_str (netid) + ) +); + +TRACE_EVENT (rpcb_unregister, +TP_PROTO ( + u32 program, + u32 version, + const char *netid + ), + +TP_ARGS (program, version, netid), + +TP_STRUCT__entry ( + __field(unsigned int, program) + __field(unsigned int, version) +__string (netid, netid) + ), + +TP_fast_assign ( + __entry->program = program; + __entry->version = version; + __assign_str(netid, netid); + ), + +TP_printk ("program=%u version=%u netid=%s" , + __entry->program, __entry->version,__get_str (netid) + ) +); + DECLARE_EVENT_CLASS(svc_xdr_buf_class, TP_PROTO( const struct svc_rqst *rqst,
net/sunrpc/rpcb_clnt.c +4 -20
@@ -35,10 +35,6 @@#include "netns.h" -#if IS_ENABLED (CONFIG_SUNRPC_DEBUG ) -# define RPCDBG_FACILITY RPCDBG_BIND -#endif -#define RPCBIND_SOCK_PATHNAME "/var/run/rpcbind.sock"#define RPCBIND_PROGRAM (100000u)
@@ -444,9 +440,7 @@ int rpcb_register(struct net *net, u32 prog, u32 vers, int prot, unsigned shortstruct sunrpc_net *sn =net_generic (net, sunrpc_net_id);bool is_set = false; -dprintk ("RPC: %sregistering (%u, %u, %d, %u) with local " -"rpcbind\n" , (port ?"" :"un" ), - prog, vers, prot, port); +trace_pmap_register (prog, vers, prot, port); msg.rpc_proc = &rpcb_procedures2[RPCBPROC_UNSET ];if (port !=0 ) {
@@ -472,11 +466,6 @@ static int rpcb_register_inet4(struct sunrpc_net *sn, map->r_addr =rpc_sockaddr2uaddr (sap,GFP_KERNEL ); -dprintk ("RPC: %sregistering [%u, %u, %s, '%s'] with " -"local rpcbind\n" , (port ?"" :"un" ), - map->r_prog, map->r_vers, - map->r_addr, map->r_netid); - msg->rpc_proc = &rpcb_procedures4[RPCBPROC_UNSET ];if (port !=0 ) { msg->rpc_proc = &rpcb_procedures4[RPCBPROC_SET ];
@@ -503,11 +492,6 @@ static int rpcb_register_inet6(struct sunrpc_net *sn, map->r_addr =rpc_sockaddr2uaddr (sap,GFP_KERNEL ); -dprintk ("RPC: %sregistering [%u, %u, %s, '%s'] with " -"local rpcbind\n" , (port ?"" :"un" ), - map->r_prog, map->r_vers, - map->r_addr, map->r_netid); - msg->rpc_proc = &rpcb_procedures4[RPCBPROC_UNSET ];if (port !=0 ) { msg->rpc_proc = &rpcb_procedures4[RPCBPROC_SET ];
@@ -524,9 +508,7 @@ static int rpcb_unregister_all_protofamilies(struct sunrpc_net *sn, {struct rpcbind_args *map = msg->rpc_argp; -dprintk ("RPC: unregistering [%u, %u, '%s'] with " -"local rpcbind\n" , - map->r_prog, map->r_vers, map->r_netid); +trace_rpcb_unregister (map->r_prog, map->r_vers, map->r_netid); map->r_addr ="" ; msg->rpc_proc = &rpcb_procedures4[RPCBPROC_UNSET ];
@@ -598,6 +580,8 @@ int rpcb_v4_register(struct net *net, const u32 program, const u32 version,if (address ==NULL )return rpcb_unregister_all_protofamilies (sn, &msg); +trace_rpcb_register (map.r_prog, map.r_vers, map.r_addr, map.r_netid); +switch (address->sa_family) {case AF_INET :return rpcb_register_inet4 (sn, address, &msg);
PATCH 20/22 SUNRPC: Clean up RPC scheduler tracepoints
In reply to Chuck LeverRemove several redundant dprintk call sites, and replace a couple of potentially useful ones with tracepoints.
--- include/trace/events/sunrpc.h | 2 ++ net/sunrpc/sched.c | 15 +++------------ 2 files changed, 5 insertions(+), 12 deletions(-)
include/trace/events/sunrpc.h +2 -0
@@ -387,6 +387,8 @@ DECLARE_EVENT_CLASS(rpc_task_running,DEFINE_RPC_RUNNING_EVENT (begin);DEFINE_RPC_RUNNING_EVENT (run_action); +DEFINE_RPC_RUNNING_EVENT (sync_sleep); +DEFINE_RPC_RUNNING_EVENT (sync_wake);DEFINE_RPC_RUNNING_EVENT (complete);DEFINE_RPC_RUNNING_EVENT (signalled);DEFINE_RPC_RUNNING_EVENT (end);
net/sunrpc/sched.c +3 -12
@@ -885,9 +885,6 @@ static void __rpc_execute(struct rpc_task *task)int task_is_async =RPC_IS_ASYNC (task);int status =0 ; -dprintk ("RPC: %5u __rpc_execute flags=0x%x\n" , - task->tk_pid, task->tk_flags); -WARN_ON_ONCE (RPC_IS_QUEUED (task));if (RPC_IS_QUEUED (task))return ;
@@ -947,7 +944,7 @@ static void __rpc_execute(struct rpc_task *task)return ;/* sync task: sleep here */ -dprintk ("RPC: %5u sync task going to sleep\n" , task->tk_pid); +trace_rpc_task_sync_sleep (task, task->tk_action); status =out_of_line_wait_on_bit (&task->tk_runstate,RPC_TASK_QUEUED , rpc_wait_bit_killable,TASK_KILLABLE );
@@ -963,11 +960,9 @@ static void __rpc_execute(struct rpc_task *task) task->tk_rpc_status = -ERESTARTSYS ;rpc_exit (task, -ERESTARTSYS ); } -dprintk ("RPC: %5u sync task resuming\n" , task->tk_pid); +trace_rpc_task_sync_wake (task, task->tk_action); } -dprintk ("RPC: %5u return %d, status %d\n" , task->tk_pid, status, - task->tk_status);/* Release all resources associated with the task */ rpc_release_task (task); }
@@ -1146,10 +1141,8 @@ static void rpc_free_task(struct rpc_task *task)put_rpccred (task->tk_op_cred);rpc_release_calldata (task->tk_ops, task->tk_calldata); -if (tk_flags &RPC_TASK_DYNAMIC ) { -dprintk ("RPC: %5u freeing task\n" , task->tk_pid); +if (tk_flags &RPC_TASK_DYNAMIC )mempool_free (task, rpc_task_mempool); - } }static void rpc_async_release (struct work_struct *work)
@@ -1203,8 +1196,6 @@ EXPORT_SYMBOL_GPL(rpc_put_task_async);static void rpc_release_task (struct rpc_task *task) { -dprintk ("RPC: %5u release task\n" , task->tk_pid); -WARN_ON_ONCE (RPC_IS_QUEUED (task));rpc_release_resources_task (task);
PATCH 21/22 SUNRPC: Remove dprintk call sites in RPC queuing functions
In reply to Chuck LeverRemove redundant call sites or call sites that are already covered by tracepoints.
--- include/trace/events/sunrpc.h | 1 + net/sunrpc/sched.c | 22 +--------------------- 2 files changed, 2 insertions(+), 21 deletions(-)
include/trace/events/sunrpc.h +1 -0
@@ -390,6 +390,7 @@ DEFINE_RPC_RUNNING_EVENT(run_action);DEFINE_RPC_RUNNING_EVENT (sync_sleep);DEFINE_RPC_RUNNING_EVENT (sync_wake);DEFINE_RPC_RUNNING_EVENT (complete); +DEFINE_RPC_RUNNING_EVENT (timeout);DEFINE_RPC_RUNNING_EVENT (signalled);DEFINE_RPC_RUNNING_EVENT (end);
net/sunrpc/sched.c +1 -21
@@ -85,7 +85,6 @@ __rpc_disable_timer(struct rpc_wait_queue *queue, struct rpc_task *task) {if (list_empty (&task->u.tk_wait.timer_list))return ; -dprintk ("RPC: %5u disabling timer\n" , task->tk_pid); task->tk_timeout =0 ;list_del (&task->u.tk_wait.timer_list);if (list_empty (&queue->timer_list.list))
@@ -111,9 +110,6 @@ static void __rpc_add_timer(struct rpc_wait_queue *queue, struct rpc_task *task, unsigned long timeout) { - dprintk("RPC: %5u setting alarm for %u ms\n", - task->tk_pid, jiffies_to_msecs(timeout - jiffies)); - task->tk_timeout = timeout;if (list_empty (&queue->timer_list.list) ||time_before (timeout, queue->timer_list.expires))rpc_set_queue_timer (queue, timeout);
@@ -216,9 +212,6 @@ static void __rpc_add_wait_queue(struct rpc_wait_queue *queue,/* barrier matches the read in rpc_wake_up_task_queue_locked() */ smp_wmb ();rpc_set_queued (task); - -dprintk ("RPC: %5u added to queue %p \"%s\"\n" , - task->tk_pid, queue,rpc_qname (queue)); }/*
@@ -241,8 +234,6 @@ static void __rpc_remove_wait_queue(struct rpc_wait_queue *queue, struct rpc_taselse list_del (&task->u.tk_wait.list); queue->qlen--; -dprintk ("RPC: %5u removed from queue %p \"%s\"\n" , - task->tk_pid, queue,rpc_qname (queue)); }static void __rpc_init_priority_wait_queue (struct rpc_wait_queue *queue,const char *qname,unsigned char nr_queues)
@@ -382,13 +373,9 @@ static void __rpc_do_sleep_on_priority(struct rpc_wait_queue *q,struct rpc_task *task, unsigned char queue_priority) { - dprintk("RPC: %5u sleep_on(queue \"%s\" time %lu)\n", - task->tk_pid, rpc_qname(q), jiffies); -trace_rpc_task_sleep (task ,q );__rpc_add_wait_queue (q, task, queue_priority); - } static void __rpc_sleep_on_priority(struct rpc_wait_queue *q,
@@ -510,9 +497,6 @@ static void __rpc_do_wake_up_task_on_wq(struct workqueue_struct *wq, struct rpc_wait_queue *queue, struct rpc_task *task) { - dprintk("RPC: %5u __rpc_wake_up_task (now %lu)\n", - task->tk_pid, jiffies); -/* Has the task been executed yet? If not, we cannot wake it up! */ if (!RPC_IS_ACTIVATED (task)) {printk (KERN_ERR "RPC: Inactive task (%p) being woken up!\n" , task);
@@ -524,8 +508,6 @@ static void __rpc_do_wake_up_task_on_wq(struct workqueue_struct *wq,__rpc_remove_wait_queue (queue, task);rpc_make_runnable (wq, task); - -dprintk ("RPC: __rpc_wake_up_task done\n" ); }/*
@@ -663,8 +645,6 @@ struct rpc_task *rpc_wake_up_first_on_wq(struct workqueue_struct *wq, {struct rpc_task *task =NULL ; -dprintk ("RPC: wake_up_first(%p \"%s\")\n" , - queue,rpc_qname (queue));spin_lock (&queue->lock); task =__rpc_find_next_queued (queue);if (task !=NULL )
@@ -770,7 +750,7 @@ static void __rpc_queue_timer_fn(struct work_struct *work)list_for_each_entry_safe (task, n, &queue->timer_list.list, u.tk_wait.timer_list) { timeo = task->tk_timeout;if (time_after_eq (now, timeo)) { -dprintk ("RPC: %5u timeout\n" , task->tk_pid); +trace_rpc_task_timeout (task, task->tk_action); task->tk_status = -ETIMEDOUT ;rpc_wake_up_task_queue_locked (queue, task);continue ;
PATCH 22/22 SUNRPC: Remove remaining dprintks from sched.c
In reply to Chuck LeverClean up.
--- net/sunrpc/sched.c | 10 ---------- 1 file changed, 10 deletions(-)
net/sunrpc/sched.c +0 -10
@@ -27,10 +27,6 @@#include "sunrpc.h" -#if IS_ENABLED (CONFIG_SUNRPC_DEBUG ) -#define RPCDBG_FACILITY RPCDBG_SCHED -#endif -#define CREATE_TRACE_POINTS #include <trace/events/sunrpc.h>
@@ -1065,9 +1061,6 @@ static void rpc_init_task(struct rpc_task *task, const struct rpc_task_setup *ta task->tk_action = rpc_prepare_task;rpc_init_task_statistics (task); - -dprintk ("RPC: new task initialized, procpid %u\n" , -task_pid_nr (current)); } static struct rpc_task *
@@ -1091,7 +1084,6 @@ struct rpc_task *rpc_new_task(const struct rpc_task_setup *setup_data)rpc_init_task (task, setup_data); task->tk_flags |= flags; -dprintk ("RPC: allocated task %p\n" , task);return task; }
@@ -1216,7 +1208,6 @@ static int rpciod_start(void)/* * Create the rpciod thread and wait for it to start. */ -dprintk ("RPC: creating workqueue rpciod\n" ); wq =alloc_workqueue ("rpciod" ,WQ_MEM_RECLAIM |WQ_UNBOUND ,0 );if (!wq) goto out_failed;
@@ -1241,7 +1232,6 @@ static void rpciod_stop(void)if (rpciod_workqueue ==NULL )return ; -dprintk ("RPC: destroying workqueue rpciod\n" ); wq = rpciod_workqueue; rpciod_workqueue =NULL ;
Quoted 8 lines · Chuck Lever
Hi-
This series replaces many client-side RPC dprintk call sites with tracepoints. The goals of this series are:
Quoted 8 lines
- Replace chatty dprintk call sites with tracepoints, which can handle a higher event rate, and won't get rate-limited.
- At some later point, expand the 0-64K range of RPC task IDs. Task IDs would be displayed only by tracepoints as 32-bit unsigned integers.
- Eliminate redundant tracepoints in the transport implementations.
None of the patches in this series made it into v5.9. I don't recall seeing any feedback on these either. Can they be considered for v5.10?
Quoted 37 lines
---
Chuck Lever (22):
SUNRPC: Removetrace_xprt_complete_rqst()
SUNRPC: Hoisttrace_xprtrdma_op_allocateinto generic code
SUNRPC: Remove debugging instrumentation fromxprt_release
SUNRPC: Update debugging instrumentation inxprt_do_reserve()
SUNRPC: Replacedprintk()call site inxprt_prepare_transmit
SUNRPC: Replacedprintk()call site inxs_nospace()
SUNRPC: Remove thedprint_status()macro
SUNRPC: Remove dprintk call site incall_start()
SUNRPC: Replace connect dprintk call sites with a tracepoint
SUNRPC: Mitigatecond_resched()inxprt_transmit()
SUNRPC: Addtrace_rpc_timeout_status()
SUNRPC: Tracecall_refreshevents
SUNRPC: Remove dprintk call site incall_decode
SUNRPC: Clean upcall_bind_status()observability
SUNRPC: Removerpcb_getport_asyncdprintk call sites
SUNRPC: Hoisttrace_xprtrdma_op_setportinto generic code
SUNRPC: Remove dprintk call sites in rpcbind XDR functions
SUNRPC: Remove more dprintks inrpcb_clnt.c
SUNRPC: Replace rpcbind dprintk call sites with tracepoints
SUNRPC: Clean up RPC scheduler tracepoints
SUNRPC: Remove dprintk call sites in RPC queuing functions
SUNRPC: Remove remaining dprintks fromsched.cinclude/trace/events/rpcrdma.h | 63 ------- include/trace/events/sunrpc.h | 285 ++++++++++++++++++++++++++++---- net/sunrpc/clnt.c | 75 ++------- net/sunrpc/rpcb_clnt.c | 129 +++------------ net/sunrpc/sched.c | 52 +----- net/sunrpc/xprt.c | 22 +-- net/sunrpc/xprtrdma/transport.c | 7 - net/sunrpc/xprtsock.c | 5 +- 8 files changed, 304 insertions(+), 334 deletions(-)
Quoted 2 lines
--
Chuck Lever
--
Chuck Lever