Possible NFSD patches for v5.8
Subsystems (5)
v4 cover Possible NFSD patches for v5.8
Hi Bruce-
To address merge conflicts with Anna's tree, I've rebased this series on v5.7-rc6 plus ("SUNRPC: Split the xdr_buf event class"). Only two commits were changed by this rebase:
SUNRPC: Move xpt_mutex into socket xpo_sendto methods
SUNRPC: Add more svcsock tracepoints
Feel free to make use of this version, or ignore it. :-)
Available to view:
https://git.linux-nfs.org/?p=cel/cel-2.6.git;a=shortlog;h=refs/heads/nfsd-5.8
Pull from this topic branch:
git://git.linux-nfs.org/projects/cel/cel-2.6.git nfsd-5.8
Highlights of this series:
* Remove serialization of sending RPC/RDMA Replies
* Convert the TCP socket send path to use xdr_buf::bvecs (pre-requisite for RPC-on-TLS)
* Fix svcrdma backchannel sendto return code
* Convert a number of dprintk call sites to use tracepoints
* Fix the "suggest braces around empty body in an 'else' statement" warning
Changes since v3:
* Rebased on v5.7-rc6 + ("SUNRPC: Split the xdr_buf event class")
Changes since v2:
* Rebased on v5.7-rc6
* Fixed a logic error that left XPT_DATA unset on return from svc_tcp_recvfrom()
* Broke down "SUNRPC: Refactor svc_recvfrom()" to separate clean ups from logic changes
* Some superfluous clean-ups have been redacted
* Add separate tracepoints for error cases (eg, tcp_recv and tcp_recv_err)
Changes since v1:
* Rebased on v5.7-rc5+
* Re-organized the series so changes interesting to linux-rdma appear together
* Addressed sparse warnings found by the kbuild test robot
* Included an additional minor clean-up: removal of the unused SVCRDMA_DEBUG macro
* Clarified several patch descriptions
---
Chuck Lever (33):
SUNRPC: Split the xdr_buf event class
SUNRPC: Move xpt_mutex into socket xpo_sendto methods
svcrdma: Clean up the tracing for rw_ctx_init errors
svcrdma: Clean up handling of get_rw_ctx errors
svcrdma: Trace page overruns when constructing RDMA Reads
svcrdma: trace undersized Write chunks
svcrdma: Fix backchannel return code
svcrdma: Remove backchannel dprintk call sites
svcrdma: Rename tracepoints that record header decoding errors
svcrdma: Remove the SVCRDMA_DEBUG macro
svcrdma: Displayed remote IP address should match stored address
svcrdma: Add tracepoints to report ->xpo_accept failures
SUNRPC: Remove kernel memory address from svc_xprt tracepoints
SUNRPC: Tracepoint to record errors in svc_xpo_create()
SUNRPC: Trace a few more generic svc_xprt events
SUNRPC: Remove "#include <trace/events/skb.h>"
SUNRPC: Add more svcsock tracepoints
SUNRPC: Replace dprintk call sites in TCP state change callouts
SUNRPC: Trace server-side rpcbind registration events
SUNRPC: Rename svc_sock::sk_reclen
SUNRPC: Restructure svc_tcp_recv_record()
SUNRPC: Replace dprintk() call sites in TCP receive path
SUNRPC: Refactor recvfrom path dealing with incomplete TCP receives
SUNRPC: Clean up svc_release_skb() functions
SUNRPC: Refactor svc_recvfrom()
SUNRPC: Restructure svc_udp_recvfrom()
SUNRPC: svc_show_status() macro should have enum definitions
NFSD: Add tracepoints to NFSD's duplicate reply cache
NFSD: Add tracepoints to the NFSD state management code
NFSD: Add tracepoints for monitoring NFSD callbacks
SUNRPC: Clean up request deferral tracepoints
NFSD: Squash an annoying compiler warning
NFSD: Fix improperly-formatted Doxygen comments
fs/nfsd/nfs4callback.c | 37 +- fs/nfsd/nfs4proc.c | 7 +- fs/nfsd/nfs4state.c | 63 ++-- fs/nfsd/nfscache.c | 57 +-- fs/nfsd/nfsctl.c | 26 +- fs/nfsd/state.h | 7 - fs/nfsd/trace.h | 345 +++++++++++++++++ include/linux/sunrpc/svc.h | 1 + include/linux/sunrpc/svc_rdma.h | 6 +- include/linux/sunrpc/svc_xprt.h | 6 + include/linux/sunrpc/svcsock.h | 6 +- include/trace/events/rpcrdma.h | 142 +++++-- include/trace/events/sunrpc.h | 419 +++++++++++++++++++-- net/sunrpc/svc.c | 19 +- net/sunrpc/svc_xprt.c | 52 +-- net/sunrpc/svcsock.c | 407 ++++++++++---------- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 121 ++---- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 21 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 92 ++--- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 10 +- net/sunrpc/xprtrdma/svc_rdma_transport.c | 55 ++- net/sunrpc/xprtsock.c | 12 +- 22 files changed, 1321 insertions(+), 590 deletions(-)
--
Chuck Lever
PATCH v4 1/33 SUNRPC: Split the xdr_buf event class
To help tie the recorded xdr_buf to a particular RPC transaction, the client side version of this class should display task ID information and the server side one should show the request's XID.
--- include/trace/events/sunrpc.h | 113 ++++++++++++++++++++++++----------------- net/sunrpc/clnt.c | 4 + net/sunrpc/svc_xprt.c | 4 + net/sunrpc/xprt.c | 2 - 4 files changed, 71 insertions(+), 52 deletions(-)
include/trace/events/sunrpc.h +66 -47
@@ -14,14 +14,17 @@#include <linux/net.h> #include <linux/tracepoint.h> -DECLARE_EVENT_CLASS(xdr_buf_class, +DECLARE_EVENT_CLASS(rpc_xdr_buf_class, TP_PROTO( + const struct rpc_task *task, const struct xdr_buf *xdr ), - TP_ARGS(xdr), + TP_ARGS(task, xdr), TP_STRUCT__entry( + __field(unsigned int, task_id) + __field(unsigned int, client_id) __field(const void *, head_base) __field(size_t, head_len) __field(const void *, tail_base)
@@ -31,6 +34,8 @@ DECLARE_EVENT_CLASS(xdr_buf_class, ), TP_fast_assign( + __entry->task_id = task->tk_pid; + __entry->client_id = task->tk_client->cl_clid; __entry->head_base = xdr->head[0 ].iov_base; __entry->head_len = xdr->head[0 ].iov_len; __entry->tail_base = xdr->tail[0 ].iov_base;
@@ -39,23 +44,26 @@ DECLARE_EVENT_CLASS(xdr_buf_class, __entry->msg_len = xdr->len; ), -TP_printk ("head=[%p,%zu] page=%u tail=[%p,%zu] len=%u" , +TP_printk ("task:%u@%u head=[%p,%zu] page=%u tail=[%p,%zu] len=%u" , + __entry->task_id, __entry->client_id, __entry->head_base, __entry->head_len, __entry->page_len, __entry->tail_base, __entry->tail_len, __entry->msg_len ) ); -#define DEFINE_XDRBUF_EVENT (name) \ - DEFINE_EVENT(xdr_buf_class, name, \ +#define DEFINE_RPCXDRBUF_EVENT (name) \ + DEFINE_EVENT(rpc_xdr_buf_class, \ + rpc_xdr_##name, \ TP_PROTO( \ + const struct rpc_task *task, \ const struct xdr_buf *xdr \ ), \ - TP_ARGS(xdr)) + TP_ARGS(task, xdr)) + +DEFINE_RPCXDRBUF_EVENT (sendto); +DEFINE_RPCXDRBUF_EVENT (recvfrom); +DEFINE_RPCXDRBUF_EVENT (reply_pages); -DEFINE_XDRBUF_EVENT (xprt_sendto); -DEFINE_XDRBUF_EVENT (xprt_recvfrom); -DEFINE_XDRBUF_EVENT (svc_recvfrom); -DEFINE_XDRBUF_EVENT (svc_sendto);TRACE_DEFINE_ENUM (RPC_AUTH_OK );TRACE_DEFINE_ENUM (RPC_AUTH_BADCRED );
@@ -526,43 +534,6 @@ TRACE_EVENT(rpc_xdr_alignment, ) ); -TRACE_EVENT (rpc_reply_pages, -TP_PROTO ( - const struct rpc_rqst *req - ), - -TP_ARGS (req), - -TP_STRUCT__entry ( - __field(unsigned int, task_id) - __field(unsigned int, client_id) - __field(const void *, head_base) - __field(size_t, head_len) - __field(const void *, tail_base) - __field(size_t, tail_len) - __field(unsigned int, page_len) - ), - -TP_fast_assign ( - __entry->task_id = req->rq_task->tk_pid; - __entry->client_id = req->rq_task->tk_client->cl_clid; - - __entry->head_base = req->rq_rcv_buf.head[0].iov_base; - __entry->head_len = req->rq_rcv_buf.head[0].iov_len; - __entry->page_len = req->rq_rcv_buf.page_len; - __entry->tail_base = req->rq_rcv_buf.tail[0].iov_base; - __entry->tail_len = req->rq_rcv_buf.tail[0].iov_len; - ), - -TP_printk ( -"task:%u@%u xdr=[%p,%zu]/%u/[%p,%zu]\n" , - __entry->task_id, __entry->client_id, - __entry->head_base, __entry->head_len, - __entry->page_len, - __entry->tail_base, __entry->tail_len - ) -); -/* * First define the enums in the below macros to be exported to userspace * via TRACE_DEFINE_ENUM().
@@ -990,6 +961,54 @@ TRACE_EVENT(xs_stream_read_request, __entry->copied, __entry->reclen, __entry->offset) ); + +DECLARE_EVENT_CLASS (svc_xdr_buf_class, +TP_PROTO ( + const struct svc_rqst *rqst, + const struct xdr_buf *xdr + ), + +TP_ARGS (rqst, xdr), + +TP_STRUCT__entry ( + __field(u32, xid) + __field(const void *, head_base) + __field(size_t, head_len) + __field(const void *, tail_base) + __field(size_t, tail_len) + __field(unsigned int, page_len) + __field(unsigned int, msg_len) + ), + +TP_fast_assign ( + __entry->xid = be32_to_cpu(rqst->rq_xid); + __entry->head_base = xdr->head[0].iov_base; + __entry->head_len = xdr->head[0].iov_len; + __entry->tail_base = xdr->tail[0].iov_base; + __entry->tail_len = xdr->tail[0].iov_len; + __entry->page_len = xdr->page_len; + __entry->msg_len = xdr->len; + ), + +TP_printk ("xid=0x%08x head=[%p,%zu] page=%u tail=[%p,%zu] len=%u" , + __entry->xid, + __entry->head_base, __entry->head_len, __entry->page_len, + __entry->tail_base, __entry->tail_len, __entry->msg_len + ) +); + +#define DEFINE_SVCXDRBUF_EVENT (name) \ + DEFINE_EVENT(svc_xdr_buf_class, \ + svc_xdr_##name, \ + TP_PROTO( \ + const struct svc_rqst *rqst, \ + const struct xdr_buf *xdr \ + ), \ + TP_ARGS(rqst, xdr)) + +DEFINE_SVCXDRBUF_EVENT (recvfrom); +DEFINE_SVCXDRBUF_EVENT (sendto); +#define show_rqstp_flags (flags) \ __print_flags(flags, "|", \ { (1UL << RQ_SECURE), "RQ_SECURE"}, \
net/sunrpc/clnt.c +2 -2
@@ -1270,7 +1270,7 @@ void rpc_prepare_reply_pages(struct rpc_rqst *req, struct page **pages, hdrsize +=RPC_REPHDRSIZE + req->rq_cred->cr_auth->au_ralign -1 ;xdr_inline_pages (&req->rq_rcv_buf, hdrsize <<2 , pages, base, len); -trace_rpc_reply_pages (req); +trace_rpc_xdr_reply_pages (req->rq_task, &req->rq_rcv_buf); }EXPORT_SYMBOL_GPL (rpc_prepare_reply_pages);
@@ -2531,7 +2531,7 @@ call_decode(struct rpc_task *task) goto out; req->rq_rcv_buf.len = req->rq_private_buf.len; -trace_xprt_recvfrom (&req->rq_rcv_buf); +trace_rpc_xdr_recvfrom (task, &req->rq_rcv_buf);/* Check that the softirq receive buffer is valid */ WARN_ON(memcmp(&req->rq_rcv_buf, &req->rq_private_buf,
net/sunrpc/svc_xprt.c +2 -2
@@ -812,7 +812,7 @@ static int svc_handle_xprt(struct svc_rqst *rqstp, struct svc_xprt *xprt)else len = xprt->xpt_ops->xpo_recvfrom (rqstp);if (len >0 ) -trace_svc_recvfrom (&rqstp->rq_arg); +trace_svc_xdr_recvfrom (rqstp, &rqstp->rq_arg); rqstp->rq_stime =ktime_get (); rqstp->rq_reserved = serv->sv_max_mesg;atomic_add (rqstp->rq_reserved, &xprt->xpt_reserved);
@@ -913,7 +913,7 @@ int svc_send(struct svc_rqst *rqstp) xb->len = xb->head[0 ].iov_len + xb->page_len + xb->tail[0 ].iov_len; -trace_svc_sendto (xb); +trace_svc_xdr_sendto (rqstp, xb);/* Grab mutex to serialize outgoing data. */ mutex_lock (&xprt->xpt_mutex);
net/sunrpc/xprt.c +1 -1
@@ -1460,7 +1460,7 @@ xprt_request_transmit(struct rpc_rqst *req, struct rpc_task *snd_task)*/ req->rq_ntrans++; -trace_xprt_sendto (&req->rq_snd_buf); +trace_rpc_xdr_sendto (task, &req->rq_snd_buf); connect_cookie = xprt->connect_cookie; status = xprt->ops->send_request (req);if (status !=0 ) {
PATCH v4 2/33 SUNRPC: Move xpt_mutex into socket xpo_sendto methods
In reply to Chuck LeverIt appears that the RPC/RDMA transport does not need serialization of calls to its xpo_sendto method. Move the mutex into the socket methods that still need that serialization.
Tail latencies are unambiguously better with this patch applied. fio randrw 8KB 70/30 on NFSv3, smaller numbers are better:
clat percentiles (usec):
With xpt_mutex:
r | 99.99th=[ 8848]
w | 99.99th=[ 9634]
Without xpt_mutex:
r | 99.99th=[ 8586]
w | 99.99th=[ 8979]
Serializing the construction of RPC/RDMA transport headers is not really necessary at this point, because the Linux NFS server implementation never changes its credit grant on a connection. If that should change, then svc_rdma_sendto will need to serialize access to the transport's credit grant fields.
--- include/linux/sunrpc/svc_xprt.h | 6 +++++ net/sunrpc/svc_xprt.c | 11 ++------- net/sunrpc/svcsock.c | 25 ++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 35 +++++++++++++--------------- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 10 +++----- net/sunrpc/xprtsock.c | 12 ++++++++-- 6 files changed, 64 insertions(+), 35 deletions(-)
include/linux/sunrpc/svc_xprt.h +6 -0
@@ -117,6 +117,12 @@ static inline int register_xpt_user(struct svc_xprt *xpt, struct svc_xpt_user *ureturn 0 ; } +static inline bool svc_xprt_is_dead (const struct svc_xprt *xprt) +{ +return (test_bit (XPT_DEAD , &xprt->xpt_flags) !=0 ) || + (test_bit (XPT_CLOSE , &xprt->xpt_flags) !=0 ); +} +int svc_reg_xprt_class (struct svc_xprt_class *);void svc_unreg_xprt_class (struct svc_xprt_class *); void svc_xprt_init(struct net *, struct svc_xprt_class *, struct svc_xprt *,
net/sunrpc/svc_xprt.c +3 -8
@@ -915,15 +915,10 @@ int svc_send(struct svc_rqst *rqstp) xb->tail[0 ].iov_len;trace_svc_xdr_sendto (rqstp, xb); -/* Grab mutex to serialize outgoing data. */ -mutex_lock (&xprt->xpt_mutex);trace_svc_stats_latency (rqstp); -if (test_bit (XPT_DEAD , &xprt->xpt_flags) - ||test_bit (XPT_CLOSE , &xprt->xpt_flags)) - len = -ENOTCONN ; -else - len = xprt->xpt_ops->xpo_sendto (rqstp); -mutex_unlock (&xprt->xpt_mutex); + + len = xprt->xpt_ops->xpo_sendto (rqstp); +trace_svc_send (rqstp, len);svc_xprt_release (rqstp);
net/sunrpc/svcsock.c +25 -0
@@ -506,6 +506,9 @@ static int svc_udp_recvfrom(struct svc_rqst *rqstp)* svc_udp_sendto - Send out a reply on a UDP socket * @rqstp: completed svc_rqst * +* xpt_mutex ensures @rqstp's whole message is written to the socket +* without interruption. +* * Returns the number of bytes sent, or a negative errno. */ static int svc_udp_sendto (struct svc_rqst *rqstp)
@@ -531,6 +534,11 @@ static int svc_udp_sendto(struct svc_rqst *rqstp)svc_set_cmsg_data (rqstp, cmh); +mutex_lock (&xprt->xpt_mutex); + +if (svc_xprt_is_dead (xprt)) + goto out_notconn; + err =xprt_sock_sendmsg (svsk->sk_sock, &msg, xdr,0 ,0 , &sent);xdr_free_bvec (xdr);if (err == -ECONNREFUSED ) {
@@ -538,9 +546,15 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) err =xprt_sock_sendmsg (svsk->sk_sock, &msg, xdr,0 ,0 , &sent);xdr_free_bvec (xdr); } + +mutex_unlock (&xprt->xpt_mutex);if (err <0 )return err;return sent; + +out_notconn: +mutex_unlock (&xprt->xpt_mutex); +return -ENOTCONN ; }static int svc_udp_has_wspace (struct svc_xprt *xprt)
@@ -1063,6 +1077,9 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)* svc_tcp_sendto - Send out a reply on a TCP socket * @rqstp: completed svc_rqst * +* xpt_mutex ensures @rqstp's whole message is written to the socket +* without interruption. +* * Returns the number of bytes sent, or a negative errno. */ static int svc_tcp_sendto (struct svc_rqst *rqstp)
@@ -1080,12 +1097,19 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp)svc_release_skb (rqstp); +mutex_lock (&xprt->xpt_mutex); +if (svc_xprt_is_dead (xprt)) + goto out_notconn; err =xprt_sock_sendmsg (svsk->sk_sock, &msg, xdr,0 , marker, &sent);xdr_free_bvec (xdr);if (err <0 || sent != (xdr->len +sizeof (marker))) goto out_close; +mutex_unlock (&xprt->xpt_mutex);return sent; +out_notconn: +mutex_unlock (&xprt->xpt_mutex); +return -ENOTCONN ; out_close: pr_notice("rpc-srv/tcp: %s: %s %d when sending %d bytes - shutting down socket\n", xprt->xpt_server->sv_name,
@@ -1093,6 +1117,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) (err < 0) ? err : sent, xdr->len);set_bit (XPT_CLOSE , &xprt->xpt_flags);svc_xprt_enqueue (xprt); +mutex_unlock (&xprt->xpt_mutex);return -EAGAIN ; }
net/sunrpc/xprtrdma/svc_rdma_backchannel.c +16 -19
@@ -210,34 +210,31 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst)return -ENOTCONN ; } -/* Send an RPC call on the passive end of a transport -* connection. +/** +* xprt_rdma_bc_send_request - Send a reverse-direction Call +* @rqst: rpc_rqst containing Call message to be sent +* +* Return values: +* %0 if the message was sent successfully +* %ENOTCONN if the message was not sent */ -static int -xprt_rdma_bc_send_request (struct rpc_rqst *rqst) +static int xprt_rdma_bc_send_request (struct rpc_rqst *rqst) {struct svc_xprt *sxprt = rqst->rq_xprt->bc_xprt; -struct svcxprt_rdma *rdma; +struct svcxprt_rdma *rdma = + container_of(sxprt, struct svcxprt_rdma, sc_xprt);int ret;dprintk ("svcrdma: sending bc call with xid: %08x\n" ,be32_to_cpu (rqst->rq_xid)); -mutex_lock (&sxprt->xpt_mutex); - - ret = -ENOTCONN ; - rdma = container_of(sxprt, struct svcxprt_rdma, sc_xprt); -if (!test_bit (XPT_DEAD , &sxprt->xpt_flags)) { - ret =rpcrdma_bc_send_request (rdma, rqst); -if (ret == -ENOTCONN ) -svc_close_xprt (sxprt); - } +if (test_bit (XPT_DEAD , &sxprt->xpt_flags)) +return -ENOTCONN ; -mutex_unlock (&sxprt->xpt_mutex); - -if (ret <0 ) -return ret; -return 0 ; + ret =rpcrdma_bc_send_request (rdma, rqst); +if (ret == -ENOTCONN ) +svc_close_xprt (sxprt); +return ret; } static void
net/sunrpc/xprtrdma/svc_rdma_sendto.c +4 -6
@@ -868,12 +868,10 @@ int svc_rdma_sendto(struct svc_rqst *rqstp)__be32 *p;int ret; -/* Create the RDMA response header. xprt->xpt_mutex, -* acquired in svc_send(), serializes RPC replies. The -* code path below that inserts the credit grant value -* into each transport header runs only inside this -* critical section. -*/ + ret = -ENOTCONN ; +if (svc_xprt_is_dead (xprt)) + goto err0; + ret = -ENOMEM ; sctxt =svc_rdma_send_ctxt_get (rdma);if (!sctxt)
net/sunrpc/xprtsock.c +10 -2
@@ -2548,8 +2548,16 @@ static int bc_sendto(struct rpc_rqst *req)return sent; } -/* -* The send routine. Borrows from svc_send +/** +* bc_send_request - Send a backchannel Call on a TCP socket +* @req: rpc_rqst containing Call message to be sent +* +* xpt_mutex ensures @rqstp's whole message is written to the socket +* without interruption. +* +* Return values: +* %0 if the message was sent successfully +* %ENOTCONN if the message was not sent */ static int bc_send_request (struct rpc_rqst *req) {
PATCH v4 3/33 svcrdma: Clean up the tracing for rw_ctx_init errors
In reply to Chuck Lever- De-duplicate code
- Rename the tracepoint with "_err" to allow enabling via glob
- Report the sg_cnt for the failing rw_ctx
- Fix a dumb signage issue
--- include/trace/events/rpcrdma.h | 12 +++++--- net/sunrpc/xprtrdma/svc_rdma_rw.c | 56 +++++++++++++++++++++++-------------- 2 files changed, 43 insertions(+), 25 deletions(-)
include/trace/events/rpcrdma.h +8 -4
@@ -1583,28 +1583,32 @@ DECLARE_EVENT_CLASS(svcrdma_dma_map_class,DEFINE_SVC_DMA_EVENT (dma_map_page);DEFINE_SVC_DMA_EVENT (dma_unmap_page); -TRACE_EVENT (svcrdma_dma_map_rwctx, +TRACE_EVENT (svcrdma_dma_map_rw_err,TP_PROTO ( const struct svcxprt_rdma *rdma, + unsigned int nents, int status ), -TP_ARGS (rdma, status), +TP_ARGS (rdma, nents, status),TP_STRUCT__entry ( __field(int, status) + __field(unsigned int, nents) __string(device, rdma->sc_cm_id->device->name)__string (addr, rdma->sc_xprt.xpt_remotebuf) ),TP_fast_assign ( __entry->status = status; + __entry->nents = nents; __assign_str(device, rdma->sc_cm_id->device->name); __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); ), -TP_printk ("addr=%s device=%s status=%d" , -__get_str (addr),__get_str (device), __entry->status +TP_printk ("addr=%s device=%s nents=%u status=%d" , +__get_str (addr),__get_str (device), __entry->nents, + __entry->status ) );
net/sunrpc/xprtrdma/svc_rdma_rw.c +35 -21
94 lines not on this page · raw message
PATCH v4 4/33 svcrdma: Clean up handling of get_rw_ctx errors
In reply to Chuck LeverClean up: Replace two dprintk call sites with a tracepoint.
--- include/trace/events/rpcrdma.h | 25 +++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 27 +++++++++++---------------- 2 files changed, 36 insertions(+), 16 deletions(-)
include/trace/events/rpcrdma.h +25 -0
@@ -1612,6 +1612,31 @@ TRACE_EVENT(svcrdma_dma_map_rw_err, ) ); +TRACE_EVENT (svcrdma_no_rwctx_err, +TP_PROTO ( + const struct svcxprt_rdma *rdma, + unsigned int num_sges + ), + +TP_ARGS (rdma, num_sges), + +TP_STRUCT__entry ( + __field(unsigned int, num_sges) + __string(device, rdma->sc_cm_id->device->name) +__string (addr, rdma->sc_xprt.xpt_remotebuf) + ), + +TP_fast_assign ( + __entry->num_sges = num_sges; + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + +TP_printk ("addr=%s device=%s num_sges=%d" , +__get_str (addr),__get_str (device), __entry->num_sges + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len
net/sunrpc/xprtrdma/svc_rdma_rw.c +11 -16
@@ -67,19 +67,22 @@ svc_rdma_get_rw_ctxt(struct svcxprt_rdma *rdma, unsigned int sges) ctxt =kmalloc (struct_size (ctxt, rw_first_sgl,SG_CHUNK_SIZE ),GFP_KERNEL );if (!ctxt) - goto out; + goto out_noctx;INIT_LIST_HEAD (&ctxt->rw_list); } ctxt->rw_sg_table.sgl = ctxt->rw_first_sgl;if (sg_alloc_table_chained (&ctxt->rw_sg_table, sges, ctxt->rw_sg_table.sgl, -SG_CHUNK_SIZE )) { -kfree (ctxt); - ctxt =NULL ; - } -out: +SG_CHUNK_SIZE )) + goto out_free;return ctxt; + +out_free: +kfree (ctxt); +out_noctx: +trace_svcrdma_no_rwctx_err (rdma, sges); +return NULL ; } static void svc_rdma_put_rw_ctxt(struct svcxprt_rdma *rdma,
@@ -456,7 +459,7 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, ctxt =svc_rdma_get_rw_ctxt (rdma, (write_len >>PAGE_SHIFT ) +2 );if (!ctxt) - goto out_noctx; +return -ENOMEM ;constructor (info, write_len, ctxt); ret = svc_rdma_rw_ctx_init(rdma, ctxt, seg_offset, seg_handle,
@@ -484,10 +487,6 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info,dprintk ("svcrdma: inadequate space in Write chunk (%u)\n" , info->wi_nsegs);return -E2BIG ; - -out_noctx: -dprintk ("svcrdma: no R/W ctxs available\n" ); -return -ENOMEM ; }/* Send one of an xdr_buf's kvecs by itself. To send a Reply
@@ -637,7 +636,7 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, sge_no =PAGE_ALIGN (info->ri_pageoff + len) >>PAGE_SHIFT ; ctxt =svc_rdma_get_rw_ctxt (cc->cc_rdma, sge_no);if (!ctxt) - goto out_noctx; +return -ENOMEM ; ctxt->rw_nents = sge_no; sg = ctxt->rw_sg_table.sgl;
@@ -676,10 +675,6 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, cc->cc_sqecount += ret;return 0 ; -out_noctx: -dprintk ("svcrdma: no R/W ctxs available\n" ); -return -ENOMEM ; - out_overrun:dprintk ("svcrdma: request overruns rq_pages\n" );return -EINVAL ;
PATCH v4 5/33 svcrdma: Trace page overruns when constructing RDMA Reads
In reply to Chuck LeverClean up: Replace a dprintk call site with a tracepoint.
--- include/trace/events/rpcrdma.h | 28 ++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- 2 files changed, 29 insertions(+), 1 deletion(-)
include/trace/events/rpcrdma.h +28 -0
@@ -1637,6 +1637,34 @@ TRACE_EVENT(svcrdma_no_rwctx_err, ) ); +TRACE_EVENT (svcrdma_page_overrun_err, +TP_PROTO ( + const struct svcxprt_rdma *rdma, + const struct svc_rqst *rqst, + unsigned int pageno + ), + +TP_ARGS (rdma, rqst, pageno), + +TP_STRUCT__entry ( + __field(unsigned int, pageno) + __field(u32, xid) + __string(device, rdma->sc_cm_id->device->name) +__string (addr, rdma->sc_xprt.xpt_remotebuf) + ), + +TP_fast_assign ( + __entry->pageno = pageno; + __entry->xid = __be32_to_cpu(rqst->rq_xid); + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + +TP_printk ("addr=%s device=%s xid=0x%08x pageno=%u" ,__get_str (addr), +__get_str (device), __entry->xid, __entry->pageno + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len
net/sunrpc/xprtrdma/svc_rdma_rw.c +1 -1
@@ -676,7 +676,7 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info,return 0 ; out_overrun: -dprintk ("svcrdma: request overruns rq_pages\n" ); +trace_svcrdma_page_overrun_err (cc->cc_rdma, rqstp, info->ri_pageno);return -EINVAL ; }
PATCH v4 6/33 svcrdma: trace undersized Write chunks
In reply to Chuck LeverClean up: Replace a dprintk call site.
This is the last remaining dprintk call site in svc_rdma_rw.c, so remove dprintk infrastructure as well.
--- include/trace/events/rpcrdma.h | 32 ++++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 7 ++----- 2 files changed, 34 insertions(+), 5 deletions(-)
include/trace/events/rpcrdma.h +32 -0
@@ -1665,6 +1665,38 @@ TRACE_EVENT(svcrdma_page_overrun_err, ) ); +TRACE_EVENT (svcrdma_small_wrch_err, +TP_PROTO ( + const struct svcxprt_rdma *rdma, + unsigned int remaining, + unsigned int seg_no, + unsigned int num_segs + ), + +TP_ARGS (rdma, remaining, seg_no, num_segs), + +TP_STRUCT__entry ( + __field(unsigned int, remaining) + __field(unsigned int, seg_no) + __field(unsigned int, num_segs) + __string(device, rdma->sc_cm_id->device->name) +__string (addr, rdma->sc_xprt.xpt_remotebuf) + ), + +TP_fast_assign ( + __entry->remaining = remaining; + __entry->seg_no = seg_no; + __entry->num_segs = num_segs; + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + +TP_printk ("addr=%s device=%s remaining=%u seg_no=%u num_segs=%u" , +__get_str (addr),__get_str (device), __entry->remaining, + __entry->seg_no, __entry->num_segs + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len
net/sunrpc/xprtrdma/svc_rdma_rw.c +2 -5
@@ -9,13 +9,10 @@#include <linux/sunrpc/rpc_rdma.h> #include <linux/sunrpc/svc_rdma.h> -#include <linux/sunrpc/debug.h> #include "xprt_rdma.h" #include <trace/events/rpcrdma.h> -#define RPCDBG_FACILITY RPCDBG_SVCXPRT -static void svc_rdma_write_done (struct ib_cq *cq,struct ib_wc *wc);static void svc_rdma_wc_read_done (struct ib_cq *cq,struct ib_wc *wc);
@@ -484,8 +481,8 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info,return 0 ; out_overflow: -dprintk ("svcrdma: inadequate space in Write chunk (%u)\n" , - info->wi_nsegs); +trace_svcrdma_small_wrch_err (rdma, remaining, info->wi_seg_no, + info->wi_nsegs);return -E2BIG ; }
PATCH v4 7/33 svcrdma: Fix backchannel return code
In reply to Chuck LeverWay back when I was writing the RPC/RDMA server-side backchannel code, I misread the TCP backchannel reply handler logic. When svc_tcp_recvfrom() successfully receives a backchannel reply, it does not return -EAGAIN. It sets XPT_DATA and returns zero.
Update svc_rdma_recvfrom() to return zero. Here, XPT_DATA doesn't need to be set again: it is set whenever a new message is received, behind a spin lock in a single threaded context.
Also, if handling the cb reply is not successful, the message is simply dropped. There's no special message framing to deal with as there is in the TCP case.
Now that the handle_bc_reply() return value is ignored, I've removed the dprintk call sites in the error exit of handle_bc_reply() in favor of trace points in other areas that already report the error cases.
--- include/linux/sunrpc/svc_rdma.h | 5 +--- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 38 +++++++--------------------- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 11 ++++---- 3 files changed, 17 insertions(+), 37 deletions(-)
include/linux/sunrpc/svc_rdma.h +2 -3
@@ -160,9 +160,8 @@ struct svc_rdma_send_ctxt { };/* svc_rdma_backchannel.c */ -extern int svc_rdma_handle_bc_reply (struct rpc_xprt *xprt, -__be32 *rdma_resp, -struct xdr_buf *rcvbuf); +extern void svc_rdma_handle_bc_reply (struct svc_rqst *rqstp, +struct svc_rdma_recv_ctxt *rctxt);/* svc_rdma_recvfrom.c */ extern void svc_rdma_recv_ctxts_destroy (struct svcxprt_rdma *rdma);
net/sunrpc/xprtrdma/svc_rdma_backchannel.c +10 -28
@@ -15,26 +15,25 @@#undef SVCRDMA_BACKCHANNEL_DEBUG/** -* svc_rdma_handle_bc_reply - Process incoming backchannel reply -* @xprt: controlling backchannel transport -* @rdma_resp: pointer to incoming transport header -* @rcvbuf: XDR buffer into which to decode the reply +* svc_rdma_handle_bc_reply - Process incoming backchannel Reply +* @rqstp: resources for handling the Reply +* @rctxt: Received message * -* Returns: -* %0 if @rcvbuf is filled in, xprt_complete_rqst called, -* %-EAGAIN if server should call ->recvfrom again. */ -int svc_rdma_handle_bc_reply (struct rpc_xprt *xprt,__be32 *rdma_resp, -struct xdr_buf *rcvbuf) +void svc_rdma_handle_bc_reply (struct svc_rqst *rqstp, +struct svc_rdma_recv_ctxt *rctxt) { +struct svc_xprt *sxprt = rqstp->rq_xprt; +struct rpc_xprt *xprt = sxprt->xpt_bc_xprt;struct rpcrdma_xprt *r_xprt =rpcx_to_rdmax (xprt); +struct xdr_buf *rcvbuf = &rqstp->rq_arg;struct kvec *dst, *src = &rcvbuf->head[0 ]; +__be32 *rdma_resp = rctxt->rc_recv_buf;struct rpc_rqst *req;u32 credits;size_t len;__be32 xid;__be32 *p; -int ret; p = (__be32 *)src->iov_base; len = src->iov_len;
@@ -49,14 +48,10 @@ int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, __func__, (int)len, p);#endif - ret = -EAGAIN ; -if (src->iov_len <24 ) - goto out_shortreply; -spin_lock (&xprt->queue_lock); req =xprt_lookup_rqst (xprt, xid);if (!req) - goto out_notfound; + goto out_unlock; dst = &req->rq_private_buf.head[0 ];memcpy (&req->rq_private_buf, &req->rq_rcv_buf,sizeof (struct xdr_buf ));
@@ -77,25 +72,12 @@ int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp,spin_unlock (&xprt->transport_lock);spin_lock (&xprt->queue_lock); - ret =0 ;xprt_complete_rqst (req->rq_task, rcvbuf->len);xprt_unpin_rqst (req); rcvbuf->len =0 ; out_unlock:spin_unlock (&xprt->queue_lock); -out: -return ret; - -out_shortreply: -dprintk ("svcrdma: short bc reply: xprt=%p, len=%zu\n" , - xprt, src->iov_len); - goto out; - -out_notfound: -dprintk ("svcrdma: unrecognized bc reply: xprt=%p, xid=%08x\n" , - xprt,be32_to_cpu (xid)); - goto out_unlock; }/* Send a backwards direction RPC call.
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +5 -6
@@ -878,12 +878,9 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) goto out_drop; rqstp->rq_xprt_hlen = ret; -if (svc_rdma_is_backchannel_reply (xprt, p)) { - ret =svc_rdma_handle_bc_reply (xprt->xpt_bc_xprt, p, - &rqstp->rq_arg); -svc_rdma_recv_ctxt_put (rdma_xprt, ctxt); -return ret; - } +if (svc_rdma_is_backchannel_reply (xprt, p)) + goto out_backchannel; +svc_rdma_get_inv_rkey (rdma_xprt, ctxt); p += rpcrdma_fixed_maxsz;
@@ -913,6 +910,8 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp)svc_rdma_recv_ctxt_put (rdma_xprt, ctxt);return ret; +out_backchannel: +svc_rdma_handle_bc_reply (rqstp, ctxt); out_drop:svc_rdma_recv_ctxt_put (rdma_xprt, ctxt);return 0 ;
PATCH v4 8/33 svcrdma: Remove backchannel dprintk call sites
In reply to Chuck LeverClean up.
--- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 48 +++------------------------- 1 file changed, 5 insertions(+), 43 deletions(-)
net/sunrpc/xprtrdma/svc_rdma_backchannel.c +5 -43
@@ -10,10 +10,6 @@#include "xprt_rdma.h" #include <trace/events/rpcrdma.h> -#define RPCDBG_FACILITY RPCDBG_SVCXPRT - -#undef SVCRDMA_BACKCHANNEL_DEBUG -/** * svc_rdma_handle_bc_reply - Process incoming backchannel Reply * @rqstp: resources for handling the Reply
@@ -31,33 +27,17 @@ void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp,__be32 *rdma_resp = rctxt->rc_recv_buf;struct rpc_rqst *req;u32 credits; -size_t len; -__be32 xid; -__be32 *p; - - p = (__be32 *)src->iov_base; - len = src->iov_len; - xid = *rdma_resp; - -#ifdef SVCRDMA_BACKCHANNEL_DEBUG -pr_info ("%s: xid=%08x, length=%zu\n" , - __func__,be32_to_cpu (xid), len); -pr_info ("%s: RPC/RDMA: %*ph\n" , - __func__, (int )RPCRDMA_HDRLEN_MIN , rdma_resp); -pr_info ("%s: RPC: %*ph\n" , - __func__, (int )len, p); -#endif spin_lock (&xprt->queue_lock); - req =xprt_lookup_rqst (xprt, xid); + req =xprt_lookup_rqst (xprt, *rdma_resp);if (!req) goto out_unlock; dst = &req->rq_private_buf.head[0 ];memcpy (&req->rq_private_buf, &req->rq_rcv_buf,sizeof (struct xdr_buf )); -if (dst->iov_len < len) +if (dst->iov_len < src->iov_len) goto out_unlock; -memcpy (dst->iov_base, p, len); +memcpy (dst->iov_base, src->iov_base, src->iov_len);xprt_pin_rqst (req);spin_unlock (&xprt->queue_lock);
@@ -66,7 +46,6 @@ void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, credits =1 ;/* don't deadlock */ else if (credits > r_xprt->rx_buf.rb_bc_max_requests) credits = r_xprt->rx_buf.rb_bc_max_requests; -spin_lock (&xprt->transport_lock); xprt->cwnd = credits <<RPC_CWNDSHIFT ;spin_unlock (&xprt->transport_lock);
@@ -174,10 +153,6 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) *p++ = xdr_zero; *p = xdr_zero; -#ifdef SVCRDMA_BACKCHANNEL_DEBUG -pr_info ("%s: %*ph\n" , __func__,64 , rqst->rq_buffer); -#endif - rqst->rq_xtime =ktime_get (); rc =svc_rdma_bc_sendto (rdma, rqst, ctxt);if (rc)
@@ -188,7 +163,6 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst)svc_rdma_send_ctxt_put (rdma, ctxt); drop_connection: -dprintk ("svcrdma: failed to send bc call\n" );return -ENOTCONN ; }
@@ -207,9 +181,6 @@ static int xprt_rdma_bc_send_request(struct rpc_rqst *rqst) container_of(sxprt, struct svcxprt_rdma, sc_xprt);int ret; -dprintk ("svcrdma: sending bc call with xid: %08x\n" , -be32_to_cpu (rqst->rq_xid)); -if (test_bit (XPT_DEAD , &sxprt->xpt_flags))return -ENOTCONN ;
@@ -222,8 +193,6 @@ static int xprt_rdma_bc_send_request(struct rpc_rqst *rqst)static void xprt_rdma_bc_close (struct rpc_xprt *xprt) { -dprintk ("svcrdma: %s: xprt %p\n" , __func__, xprt); -xprt_disconnect_done (xprt); xprt->cwnd =RPC_CWNDSHIFT ; }
@@ -231,8 +200,6 @@ xprt_rdma_bc_close(struct rpc_xprt *xprt)static void xprt_rdma_bc_put (struct rpc_xprt *xprt) { -dprintk ("svcrdma: %s: xprt %p\n" , __func__, xprt); -xprt_rdma_free_addresses (xprt);xprt_free (xprt); }
@@ -267,19 +234,14 @@ xprt_setup_rdma_bc(struct xprt_create *args)struct rpc_xprt *xprt;struct rpcrdma_xprt *new_xprt; -if (args->addrlen >sizeof (xprt->addr)) { -dprintk ("RPC: %s: address too large\n" , __func__); +if (args->addrlen >sizeof (xprt->addr))return ERR_PTR (-EBADF ); - } xprt =xprt_alloc (args->net,sizeof (*new_xprt),RPCRDMA_MAX_BC_REQUESTS ,RPCRDMA_MAX_BC_REQUESTS ); -if (!xprt) { -dprintk ("RPC: %s: couldn't allocate rpc_xprt\n" , - __func__); +if (!xprt)return ERR_PTR (-ENOMEM ); - } xprt->timeout = &xprt_rdma_bc_timeout;xprt_set_bound (xprt);
PATCH v4 9/33 svcrdma: Rename tracepoints that record header decoding errors
In reply to Chuck LeverClean up: Use a consistent naming convention so that these trace points can be enabled quickly via a glob.
--- include/trace/events/rpcrdma.h | 5 +++-- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 10 +++++----- 2 files changed, 8 insertions(+), 7 deletions(-)
include/trace/events/rpcrdma.h +3 -2
@@ -1355,7 +1355,7 @@ TRACE_EVENT(svcrdma_decode_rqst, show_rpcrdma_proc(__entry->proc), __entry->hdrlen) ); -TRACE_EVENT(svcrdma_decode_short, +TRACE_EVENT(svcrdma_decode_short_err, TP_PROTO( unsigned int hdrlen ),
@@ -1399,7 +1399,8 @@ DECLARE_EVENT_CLASS(svcrdma_badreq_event, );#define DEFINE_BADREQ_EVENT (name) \ - DEFINE_EVENT(svcrdma_badreq_event, svcrdma_decode_##name,\ + DEFINE_EVENT(svcrdma_badreq_event, \ + svcrdma_decode_##name##_err, \ TP_PROTO( \ __be32 *p \ ), \
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +5 -5
@@ -665,23 +665,23 @@ static int svc_rdma_xdr_decode_req(struct xdr_buf *rq_arg,return hdr_len; out_short: -trace_svcrdma_decode_short (rq_arg->len); +trace_svcrdma_decode_short_err (rq_arg->len);return -EINVAL ; out_version: -trace_svcrdma_decode_badvers (rdma_argp); +trace_svcrdma_decode_badvers_err (rdma_argp);return -EPROTONOSUPPORT ; out_drop: -trace_svcrdma_decode_drop (rdma_argp); +trace_svcrdma_decode_drop_err (rdma_argp);return 0 ; out_proc: -trace_svcrdma_decode_badproc (rdma_argp); +trace_svcrdma_decode_badproc_err (rdma_argp);return -EINVAL ; out_inval: -trace_svcrdma_decode_parse (rdma_argp); +trace_svcrdma_decode_parse_err (rdma_argp);return -EINVAL ; }
PATCH v4 10/33 svcrdma: Remove the SVCRDMA_DEBUG macro
In reply to Chuck LeverClean up: Commit d21b05f101ae ("rdma: SVCRMDA Header File") introduced the SVCRDMA_DEBUG macro, but it doesn't seem to have been used.
--- include/linux/sunrpc/svc_rdma.h | 1 - 1 file changed, 1 deletion(-)
include/linux/sunrpc/svc_rdma.h +0 -1
@@ -48,7 +48,6 @@#include <linux/sunrpc/rpc_rdma.h> #include <rdma/ib_verbs.h> #include <rdma/rdma_cm.h> -#define SVCRDMA_DEBUG /* Default and maximum inline threshold sizes */ enum {
PATCH v4 11/33 svcrdma: Displayed remote IP address should match stored address
In reply to Chuck LeverClean up: After commit 1e091c3bbf51 ("svcrdma: Ignore source port when computing DRC hash"), the IP address stored in xpt_remote always has a port number of zero. Thus, there's no need to display the port number when displaying the IP address of a remote NFS/RDMA client.
--- net/sunrpc/xprtrdma/svc_rdma_transport.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
net/sunrpc/xprtrdma/svc_rdma_transport.c +6 -1
@@ -211,7 +211,12 @@ static void handle_connect_req(struct rdma_cm_id *new_cma_id, newxprt->sc_ord = param->initiator_depth; sa = (struct sockaddr *)&newxprt->sc_cm_id->route.addr.dst_addr; -svc_xprt_set_remote (&newxprt->sc_xprt, sa,svc_addr_len (sa)); + newxprt->sc_xprt.xpt_remotelen =svc_addr_len (sa); +memcpy (&newxprt->sc_xprt.xpt_remote, sa, + newxprt->sc_xprt.xpt_remotelen); +snprintf (newxprt->sc_xprt.xpt_remotebuf, +sizeof (newxprt->sc_xprt.xpt_remotebuf) -1 ,"%pISc" , sa); +/* The remote port is arbitrary and not under the control of the * client ULP. Set it to a fixed value so that the DRC continues * to be effective after a reconnect.
PATCH v4 12/33 svcrdma: Add tracepoints to report ->xpo_accept failures
In reply to Chuck LeverFailure to accept a connection is typically due to a problem specific to a transport type. Also, ->xpo_accept returns NULL on error rather than reporting a specific problem.
So, add failure-specific tracepoints in svc_rdma_accept().
--- include/trace/events/rpcrdma.h | 38 +++++++++++++++++++++++++++++- net/sunrpc/xprtrdma/svc_rdma_transport.c | 27 +++++++++------------ 2 files changed, 48 insertions(+), 17 deletions(-)
include/trace/events/rpcrdma.h +37 -1
@@ -1309,9 +1309,45 @@ DECLARE_EVENT_CLASS(svcrdma_xprt_event, TP_ARGS(xprt))DEFINE_XPRT_EVENT (accept); -DEFINE_XPRT_EVENT (fail);DEFINE_XPRT_EVENT (free); +DECLARE_EVENT_CLASS (svcrdma_accept_class, +TP_PROTO ( + const struct svcxprt_rdma *rdma, + long status + ), + +TP_ARGS (rdma, status), + +TP_STRUCT__entry ( + __field(long, status) +__string (addr, rdma->sc_xprt.xpt_remotebuf) + ), + +TP_fast_assign ( + __entry->status = status; + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + +TP_printk ("addr=%s status=%ld" , +__get_str (addr), __entry->status + ) +); + +#define DEFINE_ACCEPT_EVENT (name) \ + DEFINE_EVENT(svcrdma_accept_class, svcrdma_##name##_err, \ + TP_PROTO( \ + const struct svcxprt_rdma *rdma, \ + long status \ + ), \ + TP_ARGS(rdma, status)) + +DEFINE_ACCEPT_EVENT (pd); +DEFINE_ACCEPT_EVENT (qp); +DEFINE_ACCEPT_EVENT (fabric); +DEFINE_ACCEPT_EVENT (initdepth); +DEFINE_ACCEPT_EVENT (accept); +TRACE_DEFINE_ENUM (RDMA_MSG );TRACE_DEFINE_ENUM (RDMA_NOMSG );TRACE_DEFINE_ENUM (RDMA_MSGP );
net/sunrpc/xprtrdma/svc_rdma_transport.c +11 -16
@@ -410,9 +410,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)if (!newxprt)return NULL ; -dprintk ("svcrdma: newxprt from accept queue = %p, cm_id=%p\n" , - newxprt, newxprt->sc_cm_id); - dev = newxprt->sc_cm_id->device; newxprt->sc_port_num = newxprt->sc_cm_id->port_num;
@@ -448,21 +445,17 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) newxprt->sc_pd =ib_alloc_pd (dev,0 );if (IS_ERR (newxprt->sc_pd)) { -dprintk ("svcrdma: error creating PD for connect request\n" ); +trace_svcrdma_pd_err (newxprt,PTR_ERR (newxprt->sc_pd)); goto errout; } newxprt->sc_sq_cq =ib_alloc_cq_any (dev, newxprt, newxprt->sc_sq_depth,IB_POLL_WORKQUEUE ); -if (IS_ERR (newxprt->sc_sq_cq)) { -dprintk ("svcrdma: error creating SQ CQ for connect request\n" ); +if (IS_ERR (newxprt->sc_sq_cq)) goto errout; - } newxprt->sc_rq_cq =ib_alloc_cq_any (dev, newxprt, rq_depth,IB_POLL_WORKQUEUE ); -if (IS_ERR (newxprt->sc_rq_cq)) { -dprintk ("svcrdma: error creating RQ CQ for connect request\n" ); +if (IS_ERR (newxprt->sc_rq_cq)) goto errout; - }memset (&qp_attr,0 ,sizeof qp_attr); qp_attr.event_handler = qp_event_handler;
@@ -486,7 +479,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) ret =rdma_create_qp (newxprt->sc_cm_id, newxprt->sc_pd, &qp_attr);if (ret) { -dprintk ("svcrdma: failed to create QP, ret=%d\n" , ret); +trace_svcrdma_qp_err (newxprt, ret); goto errout; } newxprt->sc_qp = newxprt->sc_cm_id->qp;
@@ -494,8 +487,10 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)if (!(dev->attrs.device_cap_flags &IB_DEVICE_MEM_MGT_EXTENSIONS )) newxprt->sc_snd_w_inv = false;if (!rdma_protocol_iwarp (dev, newxprt->sc_port_num) && - !rdma_ib_or_roce (dev, newxprt->sc_port_num)) + !rdma_ib_or_roce (dev, newxprt->sc_port_num)) { +trace_svcrdma_fabric_err (newxprt, -EINVAL ); goto errout; + }if (!svc_rdma_post_recvs (newxprt)) goto errout;
@@ -517,15 +512,17 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) conn_param.initiator_depth =min_t (int, newxprt->sc_ord, dev->attrs.max_qp_init_rd_atom);if (!conn_param.initiator_depth) { -dprintk ("svcrdma: invalid ORD setting\n" ); ret = -EINVAL ; +trace_svcrdma_initdepth_err (newxprt, ret); goto errout; } conn_param.private_data = &pmsg; conn_param.private_data_len =sizeof (pmsg); ret =rdma_accept (newxprt->sc_cm_id, &conn_param); -if (ret) +if (ret) { +trace_svcrdma_accept_err (newxprt, ret); goto errout; + }#if IS_ENABLED (CONFIG_SUNRPC_DEBUG )dprintk ("svcrdma: new connection %p accepted:\n" , newxprt);
@@ -544,8 +541,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)return &newxprt->sc_xprt; errout: -dprintk ("svcrdma: failure accepting new connection rc=%d.\n" , ret); -trace_svcrdma_xprt_fail (&newxprt->sc_xprt);/* Take a reference in case the DTO handler runs */ svc_xprt_get (&newxprt->sc_xprt);if (newxprt->sc_qp && !IS_ERR (newxprt->sc_qp))
PATCH v4 13/33 SUNRPC: Remove kernel memory address from svc_xprt tracepoints
In reply to Chuck LeverClean up: The xprt=%p was meant to distinguish events from different transports, but the addr=%s does that just as well and does not expose kernel memory addresses.
--- include/trace/events/sunrpc.h | 27 +++++++-------------------- 1 file changed, 7 insertions(+), 20 deletions(-)
include/trace/events/sunrpc.h +7 -20
@@ -1192,22 +1192,19 @@ TRACE_EVENT(svc_xprt_do_enqueue,TP_ARGS (xprt, rqst),TP_STRUCT__entry ( - __field(struct svc_xprt *, xprt) __field(int, pid) __field(unsigned long, flags)__string (addr, xprt->xpt_remotebuf) ),TP_fast_assign ( - __entry->xprt = xprt; __entry->pid = rqst? rqst->rq_task->pid : 0; __entry->flags = xprt->xpt_flags;__assign_str (addr, xprt->xpt_remotebuf); ), -TP_printk ("xprt=%p addr=%s pid=%d flags=%s" , - __entry->xprt,__get_str (addr), - __entry->pid,show_svc_xprt_flags (__entry->flags)) +TP_printk ("addr=%s pid=%d flags=%s" ,__get_str (addr), + __entry->pid,show_svc_xprt_flags (__entry->flags)) ); DECLARE_EVENT_CLASS(svc_xprt_event,
@@ -1216,20 +1213,17 @@ DECLARE_EVENT_CLASS(svc_xprt_event,TP_ARGS (xprt), TP_STRUCT__entry( - __field(struct svc_xprt *, xprt) __field(unsigned long, flags) __string(addr, xprt->xpt_remotebuf) ),TP_fast_assign ( - __entry->xprt = xprt; __entry->flags = xprt->xpt_flags;__assign_str (addr, xprt->xpt_remotebuf); ), -TP_printk ("xprt=%p addr=%s flags=%s" , - __entry->xprt,__get_str (addr), -show_svc_xprt_flags (__entry->flags)) +TP_printk ("addr=%s flags=%s" ,__get_str (addr), +show_svc_xprt_flags (__entry->flags)) ); DEFINE_EVENT(svc_xprt_event, svc_xprt_no_write_space,
@@ -1242,24 +1236,20 @@ TRACE_EVENT(svc_xprt_dequeue,TP_ARGS (rqst),TP_STRUCT__entry ( - __field(struct svc_xprt *, xprt) __field(unsigned long, flags)__field (unsigned long ,wakeup ) __string(addr, rqst->rq_xprt->xpt_remotebuf) ),TP_fast_assign ( - __entry->xprt = rqst->rq_xprt; __entry->flags = rqst->rq_xprt->xpt_flags; __entry->wakeup =ktime_to_us (ktime_sub (ktime_get (), rqst->rq_qtime));__assign_str (addr, rqst->rq_xprt->xpt_remotebuf); ), -TP_printk ("xprt=%p addr=%s flags=%s wakeup-us=%lu" , - __entry->xprt,__get_str (addr), -show_svc_xprt_flags (__entry->flags), - __entry->wakeup) +TP_printk ("addr=%s flags=%s wakeup-us=%lu" ,__get_str (addr), +show_svc_xprt_flags (__entry->flags), __entry->wakeup) ); TRACE_EVENT(svc_wake_up,
@@ -1284,21 +1274,18 @@ TRACE_EVENT(svc_handle_xprt,TP_ARGS (xprt, len),TP_STRUCT__entry ( - __field(struct svc_xprt *, xprt) __field(int, len) __field(unsigned long, flags)__string (addr, xprt->xpt_remotebuf) ),TP_fast_assign ( - __entry->xprt = xprt; __entry->len = len; __entry->flags = xprt->xpt_flags;__assign_str (addr, xprt->xpt_remotebuf); ), -TP_printk ("xprt=%p addr=%s len=%d flags=%s" , - __entry->xprt,__get_str (addr), +TP_printk ("addr=%s len=%d flags=%s" ,__get_str (addr), __entry->len,show_svc_xprt_flags (__entry->flags)) );
PATCH v4 14/33 SUNRPC: Tracepoint to record errors in svc_xpo_create()
In reply to Chuck LeverCapture transport creation failures.
--- include/trace/events/sunrpc.h | 29 +++++++++++++++++++++++++++++ net/sunrpc/svc_xprt.c | 7 ++++++- 2 files changed, 35 insertions(+), 1 deletion(-)
include/trace/events/sunrpc.h +29 -0
@@ -1186,6 +1186,35 @@ DEFINE_EVENT(svc_rqst_status, svc_send, { (1UL << XPT_KILL_TEMP), "XPT_KILL_TEMP"}, \ { (1UL << XPT_CONG_CTRL), "XPT_CONG_CTRL"}) +TRACE_EVENT (svc_xprt_create_err, +TP_PROTO ( + const char *program, + const char *protocol, + struct sockaddr *sap, + const struct svc_xprt *xprt + ), + +TP_ARGS (program, protocol, sap, xprt), + +TP_STRUCT__entry ( + __field(long, error) + __string(program, program) + __string(protocol, protocol) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + +TP_fast_assign ( + __entry->error = PTR_ERR(xprt); + __assign_str(program, program); + __assign_str(protocol, protocol); + memcpy(__entry->addr, sap, sizeof(__entry->addr)); + ), + +TP_printk ("addr=%pISpc program=%s protocol=%s error=%ld" , + __entry->addr,__get_str (program),__get_str (protocol), + __entry->error) +); + TRACE_EVENT(svc_xprt_do_enqueue, TP_PROTO(struct svc_xprt *xprt, struct svc_rqst *rqst),
net/sunrpc/svc_xprt.c +6 -1
@@ -206,6 +206,7 @@ static struct svc_xprt *__svc_xpo_create(struct svc_xprt_class *xcl, .sin6_port = htons(port), };#endif +struct svc_xprt *xprt;struct sockaddr *sap;size_t len;
@@ -224,7 +225,11 @@ static struct svc_xprt *__svc_xpo_create(struct svc_xprt_class *xcl,return ERR_PTR (-EAFNOSUPPORT ); } -return xcl->xcl_ops->xpo_create (serv, net, sap, len, flags); + xprt = xcl->xcl_ops->xpo_create (serv, net, sap, len, flags); +if (IS_ERR (xprt)) +trace_svc_xprt_create_err (serv->sv_program->pg_name, + xcl->xcl_name, sap, xprt); +return xprt; }/*
PATCH v4 15/33 SUNRPC: Trace a few more generic svc_xprt events
In reply to Chuck LeverIn lieu of dprintks or tracepoints in each individual transport implementation, introduce tracepoints in the generic part of the RPC layer. These typically fire for connection lifetime events, so shouldn't contribute a lot of noise.
--- include/trace/events/rpcrdma.h | 32 ------------------------- include/trace/events/sunrpc.h | 39 ++++++++++++++++++++++++++++-- net/sunrpc/svc_xprt.c | 22 ++++------------- net/sunrpc/svcsock.c | 12 --------- net/sunrpc/xprtrdma/svc_rdma_transport.c | 21 +++------------- 5 files changed, 45 insertions(+), 81 deletions(-)
include/trace/events/rpcrdma.h +0 -32
@@ -1279,38 +1279,6 @@ TRACE_EVENT(xprtrdma_leaked_rep,** Server-side RPC/RDMA events **/ -DECLARE_EVENT_CLASS (svcrdma_xprt_event, -TP_PROTO ( - const struct svc_xprt *xprt - ), - -TP_ARGS (xprt), - -TP_STRUCT__entry ( - __field(const void *, xprt) -__string (addr, xprt->xpt_remotebuf) - ), - -TP_fast_assign ( - __entry->xprt = xprt; - __assign_str(addr, xprt->xpt_remotebuf); - ), - -TP_printk ("xprt=%p addr=%s" , - __entry->xprt,__get_str (addr) - ) -); - -#define DEFINE_XPRT_EVENT (name) \ - DEFINE_EVENT(svcrdma_xprt_event, svcrdma_xprt_##name, \ - TP_PROTO( \ - const struct svc_xprt *xprt \ - ), \ - TP_ARGS(xprt)) - -DEFINE_XPRT_EVENT (accept); -DEFINE_XPRT_EVENT (free); - DECLARE_EVENT_CLASS(svcrdma_accept_class, TP_PROTO( const struct svcxprt_rdma *rdma,
include/trace/events/sunrpc.h +36 -3
@@ -1255,9 +1255,42 @@ DECLARE_EVENT_CLASS(svc_xprt_event, show_svc_xprt_flags(__entry->flags)) ); -DEFINE_EVENT (svc_xprt_event, svc_xprt_no_write_space, - TP_PROTO(struct svc_xprt *xprt), -TP_ARGS (xprt)); +#define DEFINE_SVC_XPRT_EVENT (name) \ + DEFINE_EVENT(svc_xprt_event, svc_xprt_##name, \ + TP_PROTO( \ + struct svc_xprt *xprt \ + ), \ + TP_ARGS(xprt)) + +DEFINE_SVC_XPRT_EVENT (no_write_space); +DEFINE_SVC_XPRT_EVENT (close); +DEFINE_SVC_XPRT_EVENT (detach); +DEFINE_SVC_XPRT_EVENT (free); + +TRACE_EVENT (svc_xprt_accept, +TP_PROTO ( + const struct svc_xprt *xprt, + const char *service + ), + +TP_ARGS (xprt, service), + +TP_STRUCT__entry ( + __string(addr, xprt->xpt_remotebuf) + __string(protocol, xprt->xpt_class->xcl_name) +__string (service, service) + ), + +TP_fast_assign ( + __assign_str(addr, xprt->xpt_remotebuf); + __assign_str(protocol, xprt->xpt_class->xcl_name) + __assign_str(service, service); + ), + +TP_printk ("addr=%s protocol=%s service=%s" , +__get_str (addr),__get_str (protocol),__get_str (service) + ) +); TRACE_EVENT(svc_xprt_dequeue, TP_PROTO(struct svc_rqst *rqst),
net/sunrpc/svc_xprt.c +5 -17
@@ -153,6 +153,7 @@ static void svc_xprt_free(struct kref *kref)xprt_put (xprt->xpt_bc_xprt);if (xprt->xpt_bc_xps)xprt_switch_put (xprt->xpt_bc_xps); +trace_svc_xprt_free (xprt); xprt->xpt_ops->xpo_free (xprt);module_put (owner); }
@@ -309,15 +310,11 @@ int svc_create_xprt(struct svc_serv *serv, const char *xprt_name, {int err; -dprintk ("svc: creating transport %s[%d]\n" , xprt_name, port); err =_svc_create_xprt (serv, xprt_name, net, family, port, flags, cred);if (err == -EPROTONOSUPPORT ) {request_module ("svc%s" , xprt_name); err =_svc_create_xprt (serv, xprt_name, net, family, port, flags, cred); } -if (err <0 ) -dprintk ("svc: transport %s not found, err %d\n" , - xprt_name, -err);return err; }EXPORT_SYMBOL_GPL (svc_create_xprt);
@@ -785,7 +782,6 @@ static int svc_handle_xprt(struct svc_rqst *rqstp, struct svc_xprt *xprt)int len =0 ;if (test_bit (XPT_CLOSE , &xprt->xpt_flags)) { -dprintk ("svc_recv: found XPT_CLOSE\n" );if (test_and_clear_bit (XPT_KILL_TEMP , &xprt->xpt_flags)) xprt->xpt_ops->xpo_kill_temp_xprt (xprt);svc_delete_xprt (xprt);
@@ -804,6 +800,7 @@ static int svc_handle_xprt(struct svc_rqst *rqstp, struct svc_xprt *xprt)if (newxpt) { newxpt->xpt_cred =get_cred (xprt->xpt_cred);svc_add_new_temp_xprt (serv, newxpt); +trace_svc_xprt_accept (newxpt, serv->sv_name); }else module_put (xprt->xpt_class->xcl_owner); }else if (svc_xprt_reserve_slot (rqstp, xprt)) {
@@ -840,14 +837,6 @@ int svc_recv(struct svc_rqst *rqstp, long timeout)struct svc_serv *serv = rqstp->rq_server;int len, err; -dprintk ("svc: server %p waiting for data (to = %ld)\n" , - rqstp, timeout); - -if (rqstp->rq_xprt) -printk (KERN_ERR -"svc_recv: service %p, transport not NULL!\n" , - rqstp); - err =svc_alloc_arg (rqstp);if (err) goto out;
@@ -895,7 +884,6 @@ EXPORT_SYMBOL_GPL(svc_recv);void svc_drop (struct svc_rqst *rqstp) {trace_svc_drop (rqstp); -dprintk ("svc: xprt %p dropped request\n" , rqstp->rq_xprt);svc_xprt_release (rqstp); }EXPORT_SYMBOL_GPL (svc_drop);
@@ -1031,11 +1019,10 @@ static void svc_delete_xprt(struct svc_xprt *xprt)struct svc_serv *serv = xprt->xpt_server;struct svc_deferred_req *dr; -/* Only do this once */ if (test_and_set_bit (XPT_DEAD , &xprt->xpt_flags)) -BUG (); +return ; -dprintk ("svc: svc_delete_xprt(%p)\n" , xprt); +trace_svc_xprt_detach (xprt); xprt->xpt_ops->xpo_detach (xprt);if (xprt->xpt_bc_xprt) xprt->xpt_bc_xprt->ops->close (xprt->xpt_bc_xprt);
@@ -1056,6 +1043,7 @@ static void svc_delete_xprt(struct svc_xprt *xprt)void svc_close_xprt (struct svc_xprt *xprt) { +trace_svc_xprt_close (xprt);set_bit (XPT_CLOSE , &xprt->xpt_flags);if (test_and_set_bit (XPT_BUSY , &xprt->xpt_flags))/* someone else will have to effect the close */
net/sunrpc/svcsock.c +0 -12
@@ -727,7 +727,6 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt)int err, slen; RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]); -dprintk ("svc: tcp_accept %p sock %p\n" , svsk, sock);if (!sock)return NULL ;
@@ -1363,11 +1362,6 @@ static struct svc_xprt *svc_create_socket(struct svc_serv *serv,int newlen;int family;int val; - RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]); - -dprintk ("svc: svc_create_socket(%s, %d, %s)\n" , - serv->sv_program->pg_name, protocol, -__svc_print_addr (sin, buf,sizeof (buf)));if (protocol !=IPPROTO_UDP && protocol !=IPPROTO_TCP ) { printk(KERN_WARNING "svc: only UDP and TCP "
@@ -1427,7 +1421,6 @@ static struct svc_xprt *svc_create_socket(struct svc_serv *serv,svc_xprt_set_local (&svsk->sk_xprt, newsin, newlen);return (struct svc_xprt *)svsk; bummer: -dprintk ("svc: svc_create_socket error = %d\n" , -error);sock_release (sock);return ERR_PTR (error); }
@@ -1441,8 +1434,6 @@ static void svc_sock_detach(struct svc_xprt *xprt) struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt);struct sock *sk = svsk->sk_sk; -dprintk ("svc: svc_sock_detach(%p)\n" , svsk); -/* put back the old socket callbacks */ lock_sock (sk); sk->sk_state_change = svsk->sk_ostate;
@@ -1459,8 +1450,6 @@ static void svc_tcp_sock_detach(struct svc_xprt *xprt) { struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); -dprintk ("svc: svc_tcp_sock_detach(%p)\n" , svsk); -svc_sock_detach (xprt);if (!test_bit (XPT_LISTENER , &xprt->xpt_flags)) {
@@ -1475,7 +1464,6 @@ static void svc_tcp_sock_detach(struct svc_xprt *xprt)static void svc_sock_free (struct svc_xprt *xprt) { struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); -dprintk ("svc: svc_sock_free(%p)\n" , svsk);if (svsk->sk_sock->file)sockfd_put (svsk->sk_sock);
net/sunrpc/xprtrdma/svc_rdma_transport.c +4 -17
@@ -314,11 +314,8 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv,struct svcxprt_rdma *cma_xprt;int ret; -dprintk ("svcrdma: Creating RDMA listener\n" ); -if ((sa->sa_family !=AF_INET ) && (sa->sa_family !=AF_INET6 )) { -dprintk ("svcrdma: Address family %d is not supported.\n" , sa->sa_family); +if (sa->sa_family !=AF_INET && sa->sa_family !=AF_INET6 )return ERR_PTR (-EAFNOSUPPORT ); - } cma_xprt =svc_rdma_create_xprt (serv, net);if (!cma_xprt)return ERR_PTR (-ENOMEM );
@@ -329,7 +326,6 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv, RDMA_PS_TCP, IB_QPT_RC);if (IS_ERR (listen_id)) { ret =PTR_ERR (listen_id); -dprintk ("svcrdma: rdma_create_id failed = %d\n" , ret); goto err0; }
@@ -338,23 +334,17 @@ static struct svc_xprt *svc_rdma_create(struct svc_serv *serv,*/ #if IS_ENABLED (CONFIG_IPV6 ) ret =rdma_set_afonly (listen_id,1 ); -if (ret) { -dprintk ("svcrdma: rdma_set_afonly failed = %d\n" , ret); +if (ret) goto err1; - }#endif ret =rdma_bind_addr (listen_id, sa); -if (ret) { -dprintk ("svcrdma: rdma_bind_addr failed = %d\n" , ret); +if (ret) goto err1; - } cma_xprt->sc_cm_id = listen_id; ret =rdma_listen (listen_id,RPCRDMA_LISTEN_BACKLOG ); -if (ret) { -dprintk ("svcrdma: rdma_listen failed = %d\n" , ret); +if (ret) goto err1; - }/* * We need to use the address from the cm_id in case the
@@ -537,7 +527,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)dprintk (" ord : %d\n" , conn_param.initiator_depth);#endif -trace_svcrdma_xprt_accept (&newxprt->sc_xprt);return &newxprt->sc_xprt; errout:
@@ -578,8 +567,6 @@ static void __svc_rdma_free(struct work_struct *work) container_of(work, struct svcxprt_rdma, sc_work);struct svc_xprt *xprt = &rdma->sc_xprt; -trace_svcrdma_xprt_free (xprt); -if (rdma->sc_qp && !IS_ERR (rdma->sc_qp))ib_drain_qp (rdma->sc_qp);
PATCH v4 16/33 SUNRPC: Remove "#include <trace/events/skb.h>"
In reply to Chuck LeverClean up: Commit 850cbaddb52d ("udp: use it's own memory accounting schema") removed the last skb-related tracepoint from svcsock.c, so it is no longer necessary to include trace/events/skb.h.
--- net/sunrpc/svcsock.c | 1 - 1 file changed, 1 deletion(-)
net/sunrpc/svcsock.c +0 -1
@@ -45,7 +45,6 @@#include <net/tcp_states.h> #include <linux/uaccess.h> #include <asm/ioctls.h> -#include <trace/events/skb.h> #include <linux/sunrpc/types.h> #include <linux/sunrpc/clnt.h>
PATCH v4 17/33 SUNRPC: Add more svcsock tracepoints
In reply to Chuck LeverIn addition to tracing recently-updated socket sendto events, this commit adds a trace event class that can be used for additional svcsock-related tracepoints in subsequent patches.
--- include/trace/events/sunrpc.h | 97 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svcsock.c | 30 ++++--------- 2 files changed, 107 insertions(+), 20 deletions(-)
include/trace/events/sunrpc.h +97 -0
@@ -14,6 +14,39 @@#include <linux/net.h> #include <linux/tracepoint.h> +TRACE_DEFINE_ENUM (SOCK_STREAM ); +TRACE_DEFINE_ENUM (SOCK_DGRAM ); +TRACE_DEFINE_ENUM (SOCK_RAW ); +TRACE_DEFINE_ENUM (SOCK_RDM ); +TRACE_DEFINE_ENUM (SOCK_SEQPACKET ); +TRACE_DEFINE_ENUM (SOCK_DCCP ); +TRACE_DEFINE_ENUM (SOCK_PACKET ); + +#define show_socket_type (type) \ + __print_symbolic(type, \ + { SOCK_STREAM, "STREAM" }, \ + { SOCK_DGRAM, "DGRAM" }, \ + { SOCK_RAW, "RAW" }, \ + { SOCK_RDM, "RDM" }, \ + { SOCK_SEQPACKET, "SEQPACKET" }, \ + { SOCK_DCCP, "DCCP" }, \ + { SOCK_PACKET, "PACKET" }) + +/* This list is known to be incomplete, add new enums as needed. */ +TRACE_DEFINE_ENUM (AF_UNSPEC ); +TRACE_DEFINE_ENUM (AF_UNIX ); +TRACE_DEFINE_ENUM (AF_LOCAL ); +TRACE_DEFINE_ENUM (AF_INET ); +TRACE_DEFINE_ENUM (AF_INET6 ); + +#define rpc_show_address_family (family) \ + __print_symbolic(family, \ + { AF_UNSPEC, "AF_UNSPEC" }, \ + { AF_UNIX, "AF_UNIX" }, \ + { AF_LOCAL, "AF_LOCAL" }, \ + { AF_INET, "AF_INET" }, \ + { AF_INET6, "AF_INET6" }) + DECLARE_EVENT_CLASS(rpc_xdr_buf_class, TP_PROTO( const struct rpc_task *task,
@@ -1403,6 +1436,70 @@ DECLARE_EVENT_CLASS(svc_deferred_event,DEFINE_SVC_DEFERRED_EVENT (drop);DEFINE_SVC_DEFERRED_EVENT (revisit); +TRACE_EVENT (svcsock_new_socket, +TP_PROTO ( + const struct socket *socket + ), + +TP_ARGS (socket), + +TP_STRUCT__entry ( + __field(unsigned long, type) + __field(unsigned long, family) +__field (bool, listener) + ), + +TP_fast_assign ( + __entry->type = socket->type; + __entry->family = socket->sk->sk_family; + __entry->listener = (socket->sk->sk_state == TCP_LISTEN); + ), + +TP_printk ("type=%s family=%s%s" , +show_socket_type (__entry->type), +rpc_show_address_family (__entry->family), + __entry->listener ?" (listener)" :"" + ) +); + +DECLARE_EVENT_CLASS (svcsock_class, +TP_PROTO ( + const struct svc_xprt *xprt, + ssize_t result + ), + +TP_ARGS (xprt, result), + +TP_STRUCT__entry ( + __field(ssize_t, result) + __field(unsigned long, flags) +__string (addr, xprt->xpt_remotebuf) + ), + +TP_fast_assign ( + __entry->result = result; + __entry->flags = xprt->xpt_flags; + __assign_str(addr, xprt->xpt_remotebuf); + ), + +TP_printk ("addr=%s result=%zd flags=%s" ,__get_str (addr), + __entry->result,show_svc_xprt_flags (__entry->flags) + ) +); + +#define DEFINE_SVCSOCK_EVENT (name) \ + DEFINE_EVENT(svcsock_class, svcsock_##name, \ + TP_PROTO( \ + const struct svc_xprt *xprt, \ + ssize_t result \ + ), \ + TP_ARGS(xprt, result)) + +DEFINE_SVCSOCK_EVENT (udp_send); +DEFINE_SVCSOCK_EVENT (tcp_send); +DEFINE_SVCSOCK_EVENT (data_ready); +DEFINE_SVCSOCK_EVENT (write_space); + DECLARE_EVENT_CLASS(cache_event, TP_PROTO( const struct cache_detail *cd,
net/sunrpc/svcsock.c +10 -20
@@ -54,6 +54,8 @@#include <linux/sunrpc/stats.h> #include <linux/sunrpc/xprt.h> +#include <trace/events/sunrpc.h> +#include "socklib.h" #include "sunrpc.h"
@@ -281,13 +283,10 @@ static void svc_data_ready(struct sock *sk)struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data;if (svsk) { -dprintk ("svc: socket %p(inet %p), busy=%d\n" , - svsk, sk, -test_bit (XPT_BUSY , &svsk->sk_xprt.xpt_flags)); -/* Refer to svc_setup_socket() for details. */ rmb (); svsk->sk_odata (sk); +trace_svcsock_data_ready (&svsk->sk_xprt,0 );if (!test_and_set_bit (XPT_DATA , &svsk->sk_xprt.xpt_flags))svc_xprt_enqueue (&svsk->sk_xprt); }
@@ -301,11 +300,9 @@ static void svc_write_space(struct sock *sk)struct svc_sock *svsk = (struct svc_sock *)(sk->sk_user_data);if (svsk) { -dprintk ("svc: socket %p(inet %p), write_space busy=%d\n" , - svsk, sk,test_bit (XPT_BUSY , &svsk->sk_xprt.xpt_flags)); -/* Refer to svc_setup_socket() for details. */ rmb (); +trace_svcsock_write_space (&svsk->sk_xprt,0 ); svsk->sk_owspace (sk);svc_xprt_enqueue (&svsk->sk_xprt); }
@@ -545,6 +542,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) err =xprt_sock_sendmsg (svsk->sk_sock, &msg, xdr,0 ,0 , &sent);xdr_free_bvec (xdr); } +trace_svcsock_udp_send (xprt, err);mutex_unlock (&xprt->xpt_mutex);if (err <0 )
@@ -616,7 +614,7 @@ static struct svc_xprt_class svc_udp_class = {static void svc_udp_init (struct svc_sock *svsk,struct svc_serv *serv) { -int err, level, optname, one =1 ; +int level, optname, one =1 ;svc_xprt_init (sock_net (svsk->sk_sock->sk), &svc_udp_class, &svsk->sk_xprt, serv);
@@ -647,9 +645,8 @@ static void svc_udp_init(struct svc_sock *svsk, struct svc_serv *serv)default :BUG (); } - err =kernel_setsockopt (svsk->sk_sock, level, optname, - (char *)&one,sizeof (one)); -dprintk ("svc: kernel_setsockopt returned %d\n" , err); +kernel_setsockopt (svsk->sk_sock, level, optname, (char *)&one, +sizeof (one)); }/*
@@ -1100,6 +1097,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) goto out_notconn; err =xprt_sock_sendmsg (svsk->sk_sock, &msg, xdr,0 , marker, &sent);xdr_free_bvec (xdr); +trace_svcsock_tcp_send (xprt, err <0 ? err : sent);if (err <0 || sent != (xdr->len +sizeof (marker))) goto out_close;mutex_unlock (&xprt->xpt_mutex);
@@ -1170,13 +1168,11 @@ static void svc_tcp_init(struct svc_sock *svsk, struct svc_serv *serv)set_bit (XPT_CACHE_AUTH , &svsk->sk_xprt.xpt_flags);set_bit (XPT_CONG_CTRL , &svsk->sk_xprt.xpt_flags);if (sk->sk_state ==TCP_LISTEN ) { -dprintk ("setting up TCP socket for listening\n" );strcpy (svsk->sk_xprt.xpt_remotebuf,"listener" );set_bit (XPT_LISTENER , &svsk->sk_xprt.xpt_flags); sk->sk_data_ready = svc_tcp_listen_data_ready;set_bit (XPT_CONN , &svsk->sk_xprt.xpt_flags); }else { -dprintk ("setting up TCP socket for reading\n" ); sk->sk_state_change = svc_tcp_state_change; sk->sk_data_ready = svc_data_ready; sk->sk_write_space = svc_write_space;
@@ -1226,7 +1222,6 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv,int pmap_register = !(flags &SVC_SOCK_ANONYMOUS );int err =0 ; -dprintk ("svc: svc_setup_socket %p\n" , sock); svsk =kzalloc (sizeof (*svsk),GFP_KERNEL );if (!svsk)return ERR_PTR (-ENOMEM );
@@ -1263,12 +1258,7 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv,else svc_tcp_init (svsk, serv); -dprintk ("svc: svc_setup_socket created %p (inet %p), " -"listen %d close %d\n" , - svsk, svsk->sk_sk, -test_bit (XPT_LISTENER , &svsk->sk_xprt.xpt_flags), -test_bit (XPT_CLOSE , &svsk->sk_xprt.xpt_flags)); - +trace_svcsock_new_socket (sock);return svsk; }
PATCH v4 18/33 SUNRPC: Replace dprintk call sites in TCP state change callouts
In reply to Chuck LeverReport TCP socket state changes and accept failures via tracepoints, replacing dprintk() call sites.
No tracepoint is added in svc_tcp_listen_data_ready. There's no information available there that isn't also reported by the svcsock_new_socket and the accept failure tracepoints.
--- include/trace/events/sunrpc.h | 67 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svcsock.c | 35 ++++----------------- 2 files changed, 73 insertions(+), 29 deletions(-)
include/trace/events/sunrpc.h +67 -0
@@ -1500,6 +1500,73 @@ DEFINE_SVCSOCK_EVENT(tcp_send);DEFINE_SVCSOCK_EVENT (data_ready);DEFINE_SVCSOCK_EVENT (write_space); +TRACE_EVENT (svcsock_tcp_state, +TP_PROTO ( + const struct svc_xprt *xprt, + const struct socket *socket + ), + +TP_ARGS (xprt, socket), + +TP_STRUCT__entry ( + __field(unsigned long, socket_state) + __field(unsigned long, sock_state) + __field(unsigned long, flags) +__string (addr, xprt->xpt_remotebuf) + ), + +TP_fast_assign ( + __entry->socket_state = socket->state; + __entry->sock_state = socket->sk->sk_state; + __entry->flags = xprt->xpt_flags; + __assign_str(addr, xprt->xpt_remotebuf); + ), + +TP_printk ("addr=%s state=%s sk_state=%s flags=%s" ,__get_str (addr), +rpc_show_socket_state (__entry->socket_state), +rpc_show_sock_state (__entry->sock_state), +show_svc_xprt_flags (__entry->flags) + ) +); + +DECLARE_EVENT_CLASS (svcsock_accept_class, +TP_PROTO ( + const struct svc_xprt *xprt, + const char *service, + long status + ), + +TP_ARGS (xprt, service, status), + +TP_STRUCT__entry ( + __field(long, status) + __string(service, service) + __array(unsigned char, addr, sizeof(struct sockaddr_in6)) + ), + +TP_fast_assign ( + __entry->status = status; + __assign_str(service, service); + memcpy(__entry->addr, &xprt->xpt_local, sizeof(__entry->addr)); + ), + +TP_printk ("listener=%pISpc service=%s status=%ld" , + __entry->addr,__get_str (service), __entry->status + ) +); + +#define DEFINE_ACCEPT_EVENT (name) \ + DEFINE_EVENT(svcsock_accept_class, svcsock_##name##_err, \ + TP_PROTO( \ + const struct svc_xprt *xprt, \ + const char *service, \ + long status \ + ), \ + TP_ARGS(xprt, service, status)) + +DEFINE_ACCEPT_EVENT (accept); +DEFINE_ACCEPT_EVENT (getpeername); + DECLARE_EVENT_CLASS(cache_event, TP_PROTO( const struct cache_detail *cd,
net/sunrpc/svcsock.c +6 -29
@@ -657,9 +657,6 @@ static void svc_tcp_listen_data_ready(struct sock *sk) {struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data; -dprintk ("svc: socket %p TCP (listen) state change %d\n" , - sk, sk->sk_state); -if (svsk) {/* Refer to svc_setup_socket() for details. */ rmb ();
@@ -680,8 +677,7 @@ static void svc_tcp_listen_data_ready(struct sock *sk)if (svsk) {set_bit (XPT_CONN , &svsk->sk_xprt.xpt_flags);svc_xprt_enqueue (&svsk->sk_xprt); - }else -printk ("svc: socket %p: no user data\n" , sk); + } } }
@@ -692,15 +688,11 @@ static void svc_tcp_state_change(struct sock *sk) {struct svc_sock *svsk = (struct svc_sock *)sk->sk_user_data; -dprintk ("svc: socket %p TCP (connected) state change %d (svsk %p)\n" , - sk, sk->sk_state, sk->sk_user_data); - -if (!svsk) -printk ("svc: socket %p: no user data\n" , sk); -else { +if (svsk) {/* Refer to svc_setup_socket() for details. */ rmb (); svsk->sk_ostate (sk); +trace_svcsock_tcp_state (&svsk->sk_xprt, svsk->sk_sock);if (sk->sk_state !=TCP_ESTABLISHED ) {set_bit (XPT_CLOSE , &svsk->sk_xprt.xpt_flags);svc_xprt_enqueue (&svsk->sk_xprt);
@@ -721,7 +713,6 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt)struct socket *newsock;struct svc_sock *newsvsk;int err, slen; - RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]);if (!sock)return NULL ;
@@ -735,30 +726,18 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt)else if (err != -EAGAIN )net_warn_ratelimited ("%s: accept failed (err %d)!\n" , serv->sv_name, -err); +trace_svcsock_accept_err (xprt, serv->sv_name, err);return NULL ; }set_bit (XPT_CONN , &svsk->sk_xprt.xpt_flags); err =kernel_getpeername (newsock, sin);if (err <0 ) { -net_warn_ratelimited ("%s: peername failed (err %d)!\n" , - serv->sv_name, -err); +trace_svcsock_getpeername_err (xprt, serv->sv_name, err); goto failed;/* aborted connection or whatever */ } slen = err; -/* Ideally, we would want to reject connections from unauthorized -* hosts here, but when we get encryption, the IP of the host won't -* tell us anything. For now just warn about unpriv connections. -*/ -if (!svc_port_is_privileged (sin)) { -dprintk ("%s: connect from unprivileged port: %s\n" , - serv->sv_name, -__svc_print_addr (sin, buf,sizeof (buf))); - } -dprintk ("%s: connect from %s\n" , serv->sv_name, -__svc_print_addr (sin, buf,sizeof (buf))); -/* Reset the inherited callbacks before calling svc_setup_socket */ newsock->sk->sk_state_change = svsk->sk_ostate; newsock->sk->sk_data_ready = svsk->sk_odata;
@@ -776,10 +755,8 @@ static struct svc_xprt *svc_tcp_accept(struct svc_xprt *xprt)svc_xprt_set_remote (&newsvsk->sk_xprt, sin, slen); err =kernel_getsockname (newsock, sin); slen = err; -if (unlikely (err <0 )) { -dprintk ("svc_tcp_accept: kernel_getsockname error %d\n" , -err); +if (unlikely (err <0 )) slen = offsetof(struct sockaddr , sa_data); - }svc_xprt_set_local (&newsvsk->sk_xprt, sin, slen);if (sock_is_loopback (newsock->sk))
PATCH v4 19/33 SUNRPC: Trace server-side rpcbind registration events
In reply to Chuck Lever--- include/trace/events/sunrpc.h | 80 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svc.c | 19 ++-------- 2 files changed, 84 insertions(+), 15 deletions(-)
include/trace/events/sunrpc.h +80 -0
@@ -1600,6 +1600,86 @@ DEFINE_CACHE_EVENT(cache_entry_update);DEFINE_CACHE_EVENT (cache_entry_make_negative);DEFINE_CACHE_EVENT (cache_entry_no_listener); +DECLARE_EVENT_CLASS (register_class, +TP_PROTO ( + const char *program, + const u32 version, + const int family, + const unsigned short protocol, + const unsigned short port, + int error + ), + +TP_ARGS (program, version, family, protocol, port, error), + +TP_STRUCT__entry ( + __field(u32, version) + __field(unsigned long, family) + __field(unsigned short, protocol) + __field(unsigned short, port) + __field(int, error) +__string (program, program) + ), + +TP_fast_assign ( + __entry->version = version; + __entry->family = family; + __entry->protocol = protocol; + __entry->port = port; + __entry->error = error; + __assign_str(program, program); + ), + +TP_printk ("program=%sv%u proto=%s port=%u family=%s error=%d" , +__get_str (program), __entry->version, + __entry->protocol ==IPPROTO_UDP ?"udp" :"tcp" , + __entry->port,rpc_show_address_family (__entry->family), + __entry->error + ) +); + +#define DEFINE_REGISTER_EVENT (name) \ + DEFINE_EVENT(register_class, svc_##name, \ + TP_PROTO( \ + const char *program, \ + const u32 version, \ + const int family, \ + const unsigned short protocol, \ + const unsigned short port, \ + int error \ + ), \ + TP_ARGS(program, version, family, protocol, \ + port, error)) + +DEFINE_REGISTER_EVENT (register); +DEFINE_REGISTER_EVENT (noregister); + +TRACE_EVENT (svc_unregister, +TP_PROTO ( + const char *program, + const u32 version, + int error + ), + +TP_ARGS (program, version, error), + +TP_STRUCT__entry ( + __field(u32, version) + __field(int, error) +__string (program, program) + ), + +TP_fast_assign ( + __entry->version = version; + __entry->error = error; + __assign_str(program, program); + ), + +TP_printk ("program=%sv%u error=%d" , +__get_str (program), __entry->version, __entry->error + ) +); +#endif /* _TRACE_SUNRPC_H */ #include <trace/define_trace.h>
net/sunrpc/svc.c +4 -15
@@ -991,6 +991,7 @@ static int __svc_register(struct net *net, const char *progname,#endif } +trace_svc_register (progname, version, protocol, port, family, error);return error; }
@@ -1000,11 +1001,6 @@ int svc_rpcbind_set_version(struct net *net, unsigned short proto, unsigned short port) { - dprintk("svc: svc_register(%sv%d, %s, %u, %u)\n", - progp->pg_name, version, - proto ==IPPROTO_UDP ?"udp" :"tcp" , - port, family); -return __svc_register (net, progp->pg_name, progp->pg_prog, version, family, proto, port);
@@ -1024,11 +1020,8 @@ int svc_generic_rpcbind_set(struct net *net,return 0 ;if (vers->vs_hidden) { -dprintk ("svc: svc_register(%sv%d, %s, %u, %u)" -" (but not telling portmap)\n" , - progp->pg_name, version, - proto ==IPPROTO_UDP ?"udp" :"tcp" , - port, family); +trace_svc_noregister (progp->pg_name, version, proto, + port, family,0 );return 0 ; }
@@ -1106,8 +1099,7 @@ static void __svc_unregister(struct net *net, const u32 program, const u32 versiif (error == -EPROTONOSUPPORT ) error =rpcb_register (net, program, version,0 ,0 ); -dprintk ("svc: %s(%sv%u), error %d\n" , - __func__, progname, version, error); +trace_svc_unregister (progname, version, error); }/*
@@ -1132,9 +1124,6 @@ static void svc_unregister(const struct svc_serv *serv, struct net *net)continue ;if (progp->pg_vers[i]->vs_hidden)continue ; - -dprintk ("svc: attempting to unregister %sv%u\n" , - progp->pg_name, i);__svc_unregister (net, progp->pg_prog, i, progp->pg_name); } }
PATCH v4 20/33 SUNRPC: Rename svc_sock::sk_reclen
In reply to Chuck LeverClean up. I find the name of the svc_sock::sk_reclen field confusing, so I've changed it to better reflect its function. This field is not read directly to get the record length. Rather, it is a buffer containing a record marker that needs to be decoded.
--- include/linux/sunrpc/svcsock.h | 6 +++--- net/sunrpc/svcsock.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-)
include/linux/sunrpc/svcsock.h +3 -3
@@ -28,7 +28,7 @@ struct svc_sock {/* private TCP part */ /* On-the-wire fragment header: */ -__be32 sk_reclen; +__be32 sk_marker;/* As we receive a record, this includes the length received so * far (including the fragment header): */ u32 sk_tcplen;
@@ -41,12 +41,12 @@ struct svc_sock {static inline u32 svc_sock_reclen (struct svc_sock *svsk) { -return ntohl (svsk->sk_reclen) &RPC_FRAGMENT_SIZE_MASK ; +return be32_to_cpu (svsk->sk_marker) &RPC_FRAGMENT_SIZE_MASK ; }static inline u32 svc_sock_final_rec (struct svc_sock *svsk) { -return ntohl (svsk->sk_reclen) &RPC_LAST_STREAM_FRAGMENT ; +return be32_to_cpu (svsk->sk_marker) &RPC_LAST_STREAM_FRAGMENT ; }/*
net/sunrpc/svcsock.c +3 -3
@@ -841,7 +841,7 @@ static int svc_tcp_recv_record(struct svc_sock *svsk, struct svc_rqst *rqstp)struct kvec iov; want =sizeof (rpc_fraghdr) - svsk->sk_tcplen; - iov.iov_base = ((char *) &svsk->sk_reclen) + svsk->sk_tcplen; + iov.iov_base = ((char *)&svsk->sk_marker) + svsk->sk_tcplen; iov.iov_len = want; len =svc_recvfrom (rqstp, &iov,1 , want,0 );if (len <0 )
@@ -938,7 +938,7 @@ static void svc_tcp_fragment_received(struct svc_sock *svsk)svc_sock_final_rec (svsk) ?"final" :"nonfinal" , svc_sock_reclen(svsk)); svsk->sk_tcplen =0 ; - svsk->sk_reclen =0 ; + svsk->sk_marker = xdr_zero; }/*
@@ -1154,7 +1154,7 @@ static void svc_tcp_init(struct svc_sock *svsk, struct svc_serv *serv) sk->sk_data_ready = svc_data_ready; sk->sk_write_space = svc_write_space; - svsk->sk_reclen =0 ; + svsk->sk_marker = xdr_zero; svsk->sk_tcplen =0 ; svsk->sk_datalen =0 ;memset (&svsk->sk_pages[0 ],0 ,sizeof (svsk->sk_pages));
PATCH v4 21/33 SUNRPC: Restructure svc_tcp_recv_record()
In reply to Chuck LeverRefactor: svc_recvfrom() is going to be converted to read into bio_vecs in a moment. Unhook the only other caller, svc_tcp_recv_record(), which just wants to read the 4-byte stream record marker into a kvec.
While we're in the area, streamline this helper by straight-lining the hot path, replace dprintk call sites with tracepoints, and reduce the number of atomic bit operations in this path.
--- include/trace/events/sunrpc.h | 24 +++++++++++++++++++ net/sunrpc/svcsock.c | 51 ++++++++++++++++++++--------------------- 2 files changed, 49 insertions(+), 26 deletions(-)
include/trace/events/sunrpc.h +24 -0
@@ -1462,6 +1462,30 @@ TRACE_EVENT(svcsock_new_socket, ) ); +TRACE_EVENT (svcsock_marker, +TP_PROTO ( + const struct svc_xprt *xprt, + __be32 marker + ), + +TP_ARGS (xprt, marker), + +TP_STRUCT__entry ( + __field(unsigned int, length) + __field(bool, last) +__string (addr, xprt->xpt_remotebuf) + ), + +TP_fast_assign ( + __entry->length = be32_to_cpu(marker) & RPC_FRAGMENT_SIZE_MASK; + __entry->last = be32_to_cpu(marker) & RPC_LAST_STREAM_FRAGMENT; + __assign_str(addr, xprt->xpt_remotebuf); + ), + +TP_printk ("addr=%s length=%u%s" ,__get_str (addr), + __entry->length, __entry->last ?" (last)" :"" ) +); + DECLARE_EVENT_CLASS(svcsock_class, TP_PROTO( const struct svc_xprt *xprt,
net/sunrpc/svcsock.c +25 -26
84 lines not on this page · raw message
PATCH v4 22/33 SUNRPC: Replace dprintk() call sites in TCP receive path
In reply to Chuck Lever--- include/trace/events/sunrpc.h | 1 + net/sunrpc/svcsock.c | 16 ++-------------- 2 files changed, 3 insertions(+), 14 deletions(-)
include/trace/events/sunrpc.h +1 -0
@@ -1521,6 +1521,7 @@ DECLARE_EVENT_CLASS(svcsock_class,DEFINE_SVCSOCK_EVENT (udp_send);DEFINE_SVCSOCK_EVENT (tcp_send); +DEFINE_SVCSOCK_EVENT (tcp_recv);DEFINE_SVCSOCK_EVENT (data_ready);DEFINE_SVCSOCK_EVENT (write_space);
net/sunrpc/svcsock.c +2 -14
@@ -119,9 +119,8 @@ static void svc_release_skb(struct svc_rqst *rqstp)if (skb) {struct svc_sock *svsk = container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt); - rqstp->rq_xprt_ctxt =NULL ; -dprintk ("svc: service %p, releasing skb %p\n" , rqstp, skb); + rqstp->rq_xprt_ctxt =NULL ;skb_free_datagram_locked (svsk->sk_sk, skb); } }
@@ -132,8 +131,6 @@ static void svc_release_udp_skb(struct svc_rqst *rqstp)if (skb) { rqstp->rq_xprt_ctxt =NULL ; - -dprintk ("svc: service %p, releasing skb %p\n" , rqstp, skb);consume_skb (skb); } }
@@ -245,8 +242,6 @@ static ssize_t svc_recvfrom(struct svc_rqst *rqstp, struct kvec *iov,if (len == buflen)set_bit (XPT_DATA , &svsk->sk_xprt.xpt_flags); -dprintk ("svc: socket %p recvfrom(%p, %zu) = %zd\n" , - svsk, iov[0 ].iov_base, iov[0 ].iov_len, len);return len; }
@@ -932,9 +927,6 @@ static int copy_pages_to_kvecs(struct kvec *vec, struct page **pages, int len)static void svc_tcp_fragment_received (struct svc_sock *svsk) {/* If we have more data, signal svc_xprt_enqueue() to try again */ -dprintk ("svc: TCP %s record (%d bytes)\n" , -svc_sock_final_rec (svsk) ?"final" :"nonfinal" , -svc_sock_reclen (svsk)); svsk->sk_tcplen =0 ; svsk->sk_marker = xdr_zero; }
@@ -954,11 +946,6 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)__be32 calldir;int pnum; -dprintk ("svc: tcp_recv %p data %d conn %d close %d\n" , - svsk,test_bit (XPT_DATA , &svsk->sk_xprt.xpt_flags), -test_bit (XPT_CONN , &svsk->sk_xprt.xpt_flags), -test_bit (XPT_CLOSE , &svsk->sk_xprt.xpt_flags)); -clear_bit (XPT_DATA , &svsk->sk_xprt.xpt_flags); len =svc_tcp_read_marker (svsk, rqstp);if (len <0 )
@@ -977,6 +964,7 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)/* Now receive data */ len =svc_recvfrom (rqstp, vec, pnum, base + want, base);if (len >=0 ) { +trace_svcsock_tcp_recv (&svsk->sk_xprt, len); svsk->sk_tcplen += len; svsk->sk_datalen += len; }
PATCH v4 23/33 SUNRPC: Refactor recvfrom path dealing with incomplete TCP receives
In reply to Chuck LeverClean up: move exception processing out of the main path.
--- include/trace/events/sunrpc.h | 31 +++++++++++++++++++++++++++++++ net/sunrpc/svcsock.c | 39 +++++++++++++++++++-------------------- 2 files changed, 50 insertions(+), 20 deletions(-)
include/trace/events/sunrpc.h +31 -0
@@ -1522,9 +1522,40 @@ DECLARE_EVENT_CLASS(svcsock_class,DEFINE_SVCSOCK_EVENT (udp_send);DEFINE_SVCSOCK_EVENT (tcp_send);DEFINE_SVCSOCK_EVENT (tcp_recv); +DEFINE_SVCSOCK_EVENT (tcp_recv_eagain); +DEFINE_SVCSOCK_EVENT (tcp_recv_err);DEFINE_SVCSOCK_EVENT (data_ready);DEFINE_SVCSOCK_EVENT (write_space); +TRACE_EVENT (svcsock_tcp_recv_short, +TP_PROTO ( + const struct svc_xprt *xprt, + u32 expected, + u32 received + ), + +TP_ARGS (xprt, expected, received), + +TP_STRUCT__entry ( + __field(u32, expected) + __field(u32, received) + __field(unsigned long, flags) +__string (addr, xprt->xpt_remotebuf) + ), + +TP_fast_assign ( + __entry->expected = expected; + __entry->received = received; + __entry->flags = xprt->xpt_flags; + __assign_str(addr, xprt->xpt_remotebuf); + ), + +TP_printk ("addr=%s flags=%s expected=%u received=%u" , +__get_str (addr),show_svc_xprt_flags (__entry->flags), + __entry->expected, __entry->received + ) +); + TRACE_EVENT(svcsock_tcp_state, TP_PROTO( const struct svc_xprt *xprt,
net/sunrpc/svcsock.c +19 -20
@@ -968,23 +968,10 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) svsk->sk_tcplen += len; svsk->sk_datalen += len; } -if (len != want || !svc_sock_final_rec (svsk)) { -svc_tcp_save_pages (svsk, rqstp); -if (len <0 && len != -EAGAIN ) - goto err_delete; -if (len == want) -svc_tcp_fragment_received (svsk); -else -dprintk ("svc: incomplete TCP record (%d of %d)\n" , - (int )(svsk->sk_tcplen -sizeof (rpc_fraghdr)), -svc_sock_reclen (svsk)); - goto err_noclose; - } - -if (svsk->sk_datalen <8 ) { - svsk->sk_datalen =0 ; - goto err_delete;/* client is nuts. */ - } +if (len != want || !svc_sock_final_rec (svsk)) + goto err_incomplete; +if (svsk->sk_datalen <8 ) + goto err_nuts; rqstp->rq_arg.len = svsk->sk_datalen; rqstp->rq_arg.page_base =0 ;
@@ -1019,14 +1006,26 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)return rqstp->rq_arg.len; +err_incomplete: +svc_tcp_save_pages (svsk, rqstp); +if (len <0 && len != -EAGAIN ) + goto err_delete; +if (len == want) +svc_tcp_fragment_received (svsk); +else +trace_svcsock_tcp_recv_short (&svsk->sk_xprt, +svc_sock_reclen (svsk), + svsk->sk_tcplen -sizeof (rpc_fraghdr)); + goto err_noclose; error:if (len != -EAGAIN ) goto err_delete; -dprintk ("RPC: TCP recvfrom got EAGAIN\n" ); +trace_svcsock_tcp_recv_eagain (&svsk->sk_xprt,0 );return 0 ; +err_nuts: + svsk->sk_datalen =0 ; err_delete: -printk (KERN_NOTICE "%s: recvfrom returned errno %d\n" , - svsk->sk_xprt.xpt_server->sv_name, -len); +trace_svcsock_tcp_recv_err (&svsk->sk_xprt, len);set_bit (XPT_CLOSE , &svsk->sk_xprt.xpt_flags); err_noclose:return 0 ;/* record not complete */
PATCH v4 24/33 SUNRPC: Clean up svc_release_skb() functions
In reply to Chuck LeverRename these functions using the convention used for other xpo method entry points.
--- net/sunrpc/svcsock.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-)
net/sunrpc/svcsock.c +15 -8
@@ -109,10 +109,12 @@ static void svc_reclassify_socket(struct socket *sock) }#endif -/* -* Release an skbuff after use +/** +* svc_tcp_release_rqst - Release transport-related resources +* @rqstp: request structure with resources to be released +* */ -static void svc_release_skb (struct svc_rqst *rqstp) +static void svc_tcp_release_rqst (struct svc_rqst *rqstp) {struct sk_buff *skb = rqstp->rq_xprt_ctxt;
@@ -125,7 +127,12 @@ static void svc_release_skb(struct svc_rqst *rqstp) } } -static void svc_release_udp_skb (struct svc_rqst *rqstp) +/** +* svc_udp_release_rqst - Release transport-related resources +* @rqstp: request structure with resources to be released +* +*/ +static void svc_udp_release_rqst (struct svc_rqst *rqstp) {struct sk_buff *skb = rqstp->rq_xprt_ctxt;
@@ -521,7 +528,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp)unsigned int uninitialized_var (sent );int err; -svc_release_udp_skb (rqstp); +svc_udp_release_rqst (rqstp);svc_set_cmsg_data (rqstp, cmh);
@@ -590,7 +597,7 @@ static const struct svc_xprt_ops svc_udp_ops = { .xpo_recvfrom = svc_udp_recvfrom, .xpo_sendto = svc_udp_sendto, .xpo_read_payload = svc_sock_read_payload, - .xpo_release_rqst = svc_release_udp_skb, + .xpo_release_rqst = svc_udp_release_rqst, .xpo_detach = svc_sock_detach, .xpo_free = svc_sock_free, .xpo_has_wspace = svc_udp_has_wspace,
@@ -1053,7 +1060,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp)unsigned int uninitialized_var (sent );int err; -svc_release_skb (rqstp); +svc_tcp_release_rqst (rqstp);mutex_lock (&xprt->xpt_mutex);if (svc_xprt_is_dead (xprt))
@@ -1093,7 +1100,7 @@ static const struct svc_xprt_ops svc_tcp_ops = { .xpo_recvfrom = svc_tcp_recvfrom, .xpo_sendto = svc_tcp_sendto, .xpo_read_payload = svc_sock_read_payload, - .xpo_release_rqst = svc_release_skb, + .xpo_release_rqst = svc_tcp_release_rqst, .xpo_detach = svc_tcp_sock_detach, .xpo_free = svc_sock_free, .xpo_has_wspace = svc_tcp_has_wspace,
PATCH v4 25/33 SUNRPC: Refactor svc_recvfrom()
In reply to Chuck LeverThis function is not currently "generic" so remove the documenting comment and rename it appropriately. Its internals are converted to use bio_vecs for reading from the transport socket.
In existing typical sunrpc uses of bio_vecs, the bio_vec array is allocated dynamically. Here, instead, an array of bio_vecs is added to svc_rqst. The lifetime of this array can be greater than one call to xpo_recvfrom():
- Multiple calls to xpo_recvfrom() might be needed to read an RPC message completely.
- At some later point, rq_arg.bvecs will point to this array and it will carry the received message into svc_process().
I also expect that a future optimization will remove either the rq_vec or rq_pages array in favor of rq_bvec, thus conserving the size of struct svc_rqst.
--- include/linux/sunrpc/svc.h | 1 net/sunrpc/svcsock.c | 109 +++++++++++++++++++++++++++----------------- 2 files changed, 69 insertions(+), 41 deletions(-)
include/linux/sunrpc/svc.h +1 -0
@@ -254,6 +254,7 @@ struct svc_rqst {struct page * *rq_page_end;/* one past the last page */ struct kvec rq_vec[RPCSVC_MAXPAGES ];/* generally useful.. */ +struct bio_vec rq_bvec[RPCSVC_MAXPAGES ];__be32 rq_xid;/* transmission id */ u32 rq_prog;/* program number */
net/sunrpc/svcsock.c +68 -41
163 lines not on this page · raw message
PATCH v4 26/33 SUNRPC: Restructure svc_udp_recvfrom()
In reply to Chuck LeverClean up. At this point, we are not ready yet to support bio_vecs in the UDP transport implementation. However, we can clean up svc_udp_recvfrom() to match the tracing and straight-lining recently changes made in svc_tcp_recvfrom().
--- include/trace/events/sunrpc.h | 2 + net/sunrpc/svcsock.c | 60 ++++++++++++++++++++++++----------------- 2 files changed, 37 insertions(+), 25 deletions(-)
include/trace/events/sunrpc.h +2 -0
@@ -1520,6 +1520,8 @@ DECLARE_EVENT_CLASS(svcsock_class, TP_ARGS(xprt, result))DEFINE_SVCSOCK_EVENT (udp_send); +DEFINE_SVCSOCK_EVENT (udp_recv); +DEFINE_SVCSOCK_EVENT (udp_recv_err);DEFINE_SVCSOCK_EVENT (tcp_send);DEFINE_SVCSOCK_EVENT (tcp_recv);DEFINE_SVCSOCK_EVENT (tcp_recv_eagain);
net/sunrpc/svcsock.c +35 -25
95 lines not on this page · raw message
PATCH v4 27/33 SUNRPC: svc_show_status() macro should have enum definitions
In reply to Chuck LeverClean up: Add missing TRACE_DEFINE_ENUMs ininclude/trace/events/sunrpc.h
--- include/trace/events/sunrpc.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
include/trace/events/sunrpc.h +11 -0
@@ -1076,6 +1076,17 @@ TRACE_EVENT(svc_recv, show_rqstp_flags(__entry->flags)) ); +TRACE_DEFINE_ENUM (SVC_GARBAGE ); +TRACE_DEFINE_ENUM (SVC_SYSERR ); +TRACE_DEFINE_ENUM (SVC_VALID ); +TRACE_DEFINE_ENUM (SVC_NEGATIVE ); +TRACE_DEFINE_ENUM (SVC_OK ); +TRACE_DEFINE_ENUM (SVC_DROP ); +TRACE_DEFINE_ENUM (SVC_CLOSE ); +TRACE_DEFINE_ENUM (SVC_DENIED ); +TRACE_DEFINE_ENUM (SVC_PENDING ); +TRACE_DEFINE_ENUM (SVC_COMPLETE ); +#define svc_show_status (status) \ __print_symbolic(status, \ { SVC_GARBAGE, "SVC_GARBAGE" }, \
PATCH v4 28/33 NFSD: Add tracepoints to NFSD's duplicate reply cache
In reply to Chuck LeverTry to capture DRC failures.
Two additional clean-ups:
- Introduce Doxygen-style comments for the main entry points
- Remove a dprintk that fires for an allocation failure. This was the only dprintk in the REPCACHE class.
--- fs/nfsd/nfscache.c | 57 +++++++++++++++++++++++++++++++------------------- fs/nfsd/trace.h | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+), 22 deletions(-)
fs/nfsd/nfscache.c +35 -22
@@ -20,8 +20,7 @@#include "nfsd.h" #include "cache.h" - -#define NFSDDBG_FACILITY NFSDDBG_REPCACHE +#include "trace.h" /* * We use this value to determine the number of hash buckets from the max
@@ -323,8 +322,10 @@ nfsd_cache_key_cmp(const struct svc_cacherep *key, const struct svc_cacherep *rp, struct nfsd_net *nn) { if (key->c_key.k_xid == rp->c_key.k_xid && - key->c_key.k_csum != rp->c_key.k_csum) + key->c_key.k_csum != rp->c_key.k_csum) { ++nn->payload_misses; +trace_nfsd_drc_mismatch (nn, key, rp); + }return memcmp (&key->c_key, &rp->c_key,sizeof (key->c_key)); }
@@ -377,15 +378,22 @@ nfsd_cache_insert(struct nfsd_drc_bucket *b, struct svc_cacherep *key,return ret; } -/* +/** +* nfsd_cache_lookup - Find an entry in the duplicate reply cache +* @rqstp: Incoming Call to find +* * Try to find an entry matching the current call in the cache. When none * is found, we try to grab the oldest expired entry off the LRU list. If * a suitable one isn't there, then drop the cache_lock and allocate a * new one, then search again in case one got inserted while this thread * didn't hold the lock. +* +* Return values: +* %RC_DOIT: Process the request normally +* %RC_REPLY: Reply from cache +* %RC_DROPIT: Do not process the request further */ -int -nfsd_cache_lookup (struct svc_rqst *rqstp) +int nfsd_cache_lookup (struct svc_rqst *rqstp) {struct nfsd_net *nn =net_generic (SVC_NET (rqstp), nfsd_net_id);struct svc_cacherep *rp, *found;
@@ -399,7 +407,7 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) rqstp->rq_cacherep =NULL ;if (type ==RC_NOCACHE ) { nfsdstats.rcnocache++; -return rtn; + goto out; } csum =nfsd_cache_csum (rqstp);
@@ -409,10 +417,8 @@ nfsd_cache_lookup(struct svc_rqst *rqstp)* preallocate an entry. */ rp =nfsd_reply_cache_alloc (rqstp, csum, nn); -if (!rp) { -dprintk ("nfsd: unable to allocate DRC entry!\n" ); -return rtn; - } +if (!rp) + goto out;spin_lock (&b->cache_lock); found =nfsd_cache_insert (b, rp, nn);
@@ -431,8 +437,10 @@ nfsd_cache_lookup(struct svc_rqst *rqstp)/* go ahead and prune the cache */ prune_bucket (b, nn); - out: + +out_unlock:spin_unlock (&b->cache_lock); +out:return rtn; found_entry:
@@ -442,13 +450,13 @@ nfsd_cache_lookup(struct svc_rqst *rqstp)/* Request being processed */ if (rp->c_state ==RC_INPROG ) - goto out; + goto out_trace;/* From the hall of fame of impractical attacks: * Is this a user who tries to snoop on the cache? */ rtn =RC_DOIT ;if (!test_bit (RQ_SECURE , &rqstp->rq_flags) && rp->c_secure) - goto out; + goto out_trace;/* Compose RPC reply header */ switch (rp->c_type) {
@@ -460,7 +468,7 @@ nfsd_cache_lookup(struct svc_rqst *rqstp)break ;case RC_REPLBUFF :if (!nfsd_cache_append (rqstp, &rp->c_replvec)) - goto out;/* should not happen */ + goto out_unlock;/* should not happen */ rtn =RC_REPLY ;break ;default :
@@ -468,13 +476,19 @@ nfsd_cache_lookup(struct svc_rqst *rqstp)nfsd_reply_cache_free_locked (b, rp, nn); } - goto out; +out_trace: +trace_nfsd_drc_found (nn, rqstp, rtn); + goto out_unlock; } -/* -* Update a cache entry. This is called from nfsd_dispatch when -* the procedure has been executed and the complete reply is in -* rqstp->rq_res. +/** +* nfsd_cache_update - Update an entry in the duplicate reply cache. +* @rqstp: svc_rqst with a finished Reply +* @cachetype: which cache to update +* @statp: Reply's status code +* +* This is called from nfsd_dispatch when the procedure has been +* executed and the complete reply is in rqstp->rq_res. * * We're copying around data here rather than swapping buffers because * the toplevel loop requires max-sized buffers, which would be a waste
@@ -487,8 +501,7 @@ nfsd_cache_lookup(struct svc_rqst *rqstp)* nfsd failed to encode a reply that otherwise would have been cached. * In this case, nfsd_cache_update is called with statp == NULL. */ -void -nfsd_cache_update (struct svc_rqst *rqstp,int cachetype,__be32 *statp) +void nfsd_cache_update (struct svc_rqst *rqstp,int cachetype,__be32 *statp) {struct nfsd_net *nn =net_generic (SVC_NET (rqstp), nfsd_net_id);struct svc_cacherep *rp = rqstp->rq_cacherep;
fs/nfsd/trace.h +59 -0
65 lines not on this page · raw message
PATCH v4 29/33 NFSD: Add tracepoints to the NFSD state management code
In reply to Chuck LeverCapture obvious events and replace dprintk() call sites. Introduce infrastructure so that adding more tracepoints in this code later is simplified.
--- fs/nfsd/nfs4state.c | 52 +++++++------------- fs/nfsd/state.h | 7 --- fs/nfsd/trace.h | 133 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 152 insertions(+), 40 deletions(-)
fs/nfsd/nfs4state.c +19 -33
@@ -51,6 +51,7 @@#include "netns.h" #include "pnfs.h" #include "filecache.h" +#include "trace.h" #define NFSDDBG_FACILITY NFSDDBG_PROC
@@ -167,9 +168,6 @@ renew_client_locked(struct nfs4_client *clp)return ; } -dprintk ("renewing client (clientid %08x/%08x)\n" , - clp->cl_clientid.cl_boot, - clp->cl_clientid.cl_id);list_move_tail (&clp->cl_lru, &nn->client_lru); clp->cl_time =ktime_get_boottime_seconds (); }
@@ -1922,8 +1920,7 @@ STALE_CLIENTID(clientid_t *clid, struct nfsd_net *nn)*/ if (clid->cl_boot == (u32 )nn->boot_time)return 0 ; -dprintk ("NFSD stale clientid (%08x/%08x) boot_time %08llx\n" , - clid->cl_boot, clid->cl_id, nn->boot_time); +trace_nfsd_clid_stale (clid);return 1 ; }
@@ -3879,11 +3876,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,if (clp_used_exchangeid (conf)) goto out;if (!same_creds (&conf->cl_cred, &rqstp->rq_cred)) { -char addr_str[INET6_ADDRSTRLEN ]; -rpc_ntop ((struct sockaddr *) &conf->cl_addr, addr_str, -sizeof (addr_str)); -dprintk ("NFSD: setclientid: string in use by client " -"at %s\n" , addr_str); +trace_nfsd_clid_inuse_err (conf); goto out; } }
@@ -4076,7 +4069,6 @@ nfsd4_init_slabs(void) out_free_client_slab:kmem_cache_destroy (client_slab); out: -dprintk ("nfsd4: out of memory while initializing nfsv4\n" );return -ENOMEM ; }
@@ -4508,6 +4500,8 @@ nfsd_break_deleg_cb(struct file_lock *fl)struct nfs4_delegation *dp = (struct nfs4_delegation *)fl->fl_owner;struct nfs4_file *fp = dp->dl_stid.sc_file; +trace_nfsd_deleg_break (&dp->dl_stid.sc_stateid); +/* * We don't want the locks code to timeout the lease for us; * we'll remove it ourself if a delegation isn't returned
@@ -5018,8 +5012,7 @@ nfs4_open_delegation(struct svc_fh *fh, struct nfsd4_open *open,memcpy (&open->op_delegate_stateid, &dp->dl_stid.sc_stateid,sizeof (dp->dl_stid.sc_stateid)); -dprintk ("NFSD: delegation stateid=" STATEID_FMT "\n" , -STATEID_VAL (&dp->dl_stid.sc_stateid)); +trace_nfsd_deleg_open (&dp->dl_stid.sc_stateid); open->op_delegate_type =NFS4_OPEN_DELEGATE_READ ;nfs4_put_stid (&dp->dl_stid);return ;
@@ -5136,9 +5129,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfnfs4_open_delegation (current_fh, open, stp); nodeleg: status = nfs_ok; - -dprintk ("%s: stateid=" STATEID_FMT "\n" , __func__, -STATEID_VAL (&stp->st_stid.sc_stateid)); +trace_nfsd_deleg_none (&stp->st_stid.sc_stateid); out:/* 4.1 client trying to upgrade/downgrade delegation? */ if (open->op_delegate_type == NFS4_OPEN_DELEGATE_NONE && dp &&
@@ -5192,8 +5183,7 @@ nfsd4_renew(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,__be32 status;struct nfsd_net *nn =net_generic (SVC_NET (rqstp), nfsd_net_id); -dprintk ("process_renew(%08x/%08x): starting\n" , - clid->cl_boot, clid->cl_id); +trace_nfsd_clid_renew (clid); status =lookup_clientid (clid, cstate, nn, false);if (status) goto out;
@@ -5214,6 +5204,7 @@ nfsd4_end_grace(struct nfsd_net *nn)if (nn->grace_ended)return ; +trace_nfsd_grace_complete (nn); nn->grace_ended = true;/* * If the server goes down again right now, an NFSv4
@@ -5279,13 +5270,10 @@ nfs4_laundromat(struct nfsd_net *nn)copy_stateid_t *cps_t;int i; -dprintk ("NFSD: laundromat service - starting\n" ); -if (clients_still_reclaiming (nn)) { new_timeo =0 ; goto out; } -dprintk ("NFSD: end of grace period\n" );nfsd4_end_grace (nn);INIT_LIST_HEAD (&reaplist);
@@ -5307,8 +5295,7 @@ nfs4_laundromat(struct nfsd_net *nn)break ; }if (mark_client_expired_locked (clp)) { -dprintk ("NFSD: client in use (clientid %08x)\n" , - clp->cl_clientid.cl_id); +trace_nfsd_clid_expired (&clp->cl_clientid);continue ; }list_add (&clp->cl_lru, &reaplist);
@@ -5316,8 +5303,7 @@ nfs4_laundromat(struct nfsd_net *nn)spin_unlock (&nn->client_lock);list_for_each_safe (pos, next, &reaplist) { clp = list_entry(pos, struct nfs4_client, cl_lru); -dprintk ("NFSD: purging unused client (clientid %08x)\n" , - clp->cl_clientid.cl_id); +trace_nfsd_clid_purged (&clp->cl_clientid);list_del_init (&clp->cl_lru);expire_client (clp); }
@@ -5407,7 +5393,6 @@ laundromat_main(struct work_struct *laundry) laundromat_work); t =nfs4_laundromat (nn); -dprintk ("NFSD: laundromat_main - sleeping for %lld seconds\n" , t);queue_delayed_work (laundry_wq, &nn->laundromat_work, t*HZ ); }
@@ -5948,8 +5933,7 @@ nfs4_preprocess_seqid_op(struct nfsd4_compound_state *cstate, u32 seqid,struct nfs4_stid *s;struct nfs4_ol_stateid *stp =NULL ; -dprintk ("NFSD: %s: seqid=%d stateid = " STATEID_FMT "\n" , __func__, - seqid,STATEID_VAL (stateid)); +trace_nfsd_preprocess (seqid, stateid); *stpp =NULL ; status =nfsd4_lookup_stateid (cstate, stateid, typemask, &s, nn);
@@ -6018,9 +6002,7 @@ nfsd4_open_confirm(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, oo->oo_flags |=NFS4_OO_CONFIRMED ;nfs4_inc_and_copy_stateid (&oc->oc_resp_stateid, &stp->st_stid);mutex_unlock (&stp->st_mutex); -dprintk ("NFSD: %s: success, seqid=%d stateid=" STATEID_FMT "\n" , - __func__, oc->oc_seqid,STATEID_VAL (&stp->st_stid.sc_stateid)); - +trace_nfsd_open_confirm (oc->oc_seqid, &stp->st_stid.sc_stateid);nfsd4_client_record_create (oo->oo_owner.so_client); status = nfs_ok; put_stateid:
@@ -7072,7 +7054,7 @@ nfs4_client_to_reclaim(struct xdr_netobj name, struct xdr_netobj princhash,unsigned int strhashval;struct nfs4_client_reclaim *crp; -dprintk ("NFSD nfs4_client_to_reclaim NAME: %.*s\n" , name.len, name.data); +trace_nfsd_clid_reclaim (nn, name.len, name.data); crp =alloc_reclaim ();if (crp) { strhashval =clientstr_hashval (name);
@@ -7122,7 +7104,7 @@ nfsd4_find_reclaim_client(struct xdr_netobj name, struct nfsd_net *nn)unsigned int strhashval;struct nfs4_client_reclaim *crp =NULL ; -dprintk ("NFSD: nfs4_find_reclaim_client for name %.*s\n" , name.len, name.data); +trace_nfsd_clid_find (nn, name.len, name.data); strhashval =clientstr_hashval (name);list_for_each_entry (crp, &nn->reclaim_str_hashtbl[strhashval], cr_strhash) {
@@ -7686,6 +7668,9 @@ nfsd_recall_delegations(struct list_head *reaplist)list_for_each_entry_safe (dp, next, reaplist, dl_recall_lru) {list_del_init (&dp->dl_recall_lru); clp = dp->dl_stid.sc_client; + +trace_nfsd_deleg_recall (&dp->dl_stid.sc_stateid); +/* * We skipped all entries that had a zero dl_time before, * so we can now reset the dl_time back to 0. If a delegation
@@ -7868,6 +7853,7 @@ nfs4_state_start_net(struct net *net) goto skip_grace;printk (KERN_INFO "NFSD: starting %lld-second grace period (net %x)\n" , nn->nfsd4_grace, net->ns.inum); +trace_nfsd_grace_start (nn);queue_delayed_work (laundry_wq, &nn->laundromat_work, nn->nfsd4_grace *HZ );return 0 ;
fs/nfsd/state.h +0 -7
@@ -64,13 +64,6 @@ typedef struct {refcount_t sc_count; } copy_stateid_t; -#define STATEID_FMT "(%08x/%08x/%08x/%08x)" -#define STATEID_VAL (s) \ - (s)->si_opaque.so_clid.cl_boot, \ - (s)->si_opaque.so_clid.cl_id, \ - (s)->si_opaque.so_id, \ - (s)->si_generation -struct nfsd4_callback {struct nfs4_client *cb_clp;struct rpc_message cb_msg;
fs/nfsd/trace.h +133 -0
145 lines not on this page · raw message
PATCH v4 30/33 NFSD: Add tracepoints for monitoring NFSD callbacks
In reply to Chuck Lever--- fs/nfsd/nfs4callback.c | 37 +++++++----- fs/nfsd/nfs4state.c | 6 +- fs/nfsd/trace.h | 153 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 177 insertions(+), 19 deletions(-)
fs/nfsd/nfs4callback.c +22 -15
@@ -38,6 +38,7 @@#include "nfsd.h" #include "state.h" #include "netns.h" +#include "trace.h" #include "xdr4cb.h" #include "xdr4.h"
@@ -904,16 +905,20 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *cif (clp->cl_minorversion ==0 ) {if (!clp->cl_cred.cr_principal && - (clp->cl_cred.cr_flavor >=RPC_AUTH_GSS_KRB5 )) + (clp->cl_cred.cr_flavor >=RPC_AUTH_GSS_KRB5 )) { +trace_nfsd_cb_setup_err (clp, -EINVAL );return -EINVAL ; + } args.client_name = clp->cl_cred.cr_principal; args.prognumber = conn->cb_prog; args.protocol =XPRT_TRANSPORT_TCP ; args.authflavor = clp->cl_cred.cr_flavor; clp->cl_cb_ident = conn->cb_ident; }else { -if (!conn->cb_xprt) +if (!conn->cb_xprt) { +trace_nfsd_cb_setup_err (clp, -EINVAL );return -EINVAL ; + } clp->cl_cb_conn.cb_xprt = conn->cb_xprt; clp->cl_cb_session = ses; args.bc_xprt = conn->cb_xprt;
@@ -925,32 +930,27 @@ static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c/* Create RPC client */ client =rpc_create (&args);if (IS_ERR (client)) { -dprintk ("NFSD: couldn't create callback client: %ld\n" , -PTR_ERR (client)); +trace_nfsd_cb_setup_err (clp,PTR_ERR (client));return PTR_ERR (client); } cred =get_backchannel_cred (clp, client, ses);if (!cred) { +trace_nfsd_cb_setup_err (clp, -ENOMEM );rpc_shutdown_client (client);return -ENOMEM ; } clp->cl_cb_client = client; clp->cl_cb_cred = cred; +trace_nfsd_cb_setup (clp);return 0 ; } -static void warn_no_callback_path (struct nfs4_client *clp,int reason) -{ -dprintk ("NFSD: warning: no callback path to client %.*s: error %d\n" , - (int )clp->cl_name.len, clp->cl_name.data, reason); -} -static void nfsd4_mark_cb_down (struct nfs4_client *clp,int reason) {if (test_bit (NFSD4_CLIENT_CB_UPDATE , &clp->cl_flags))return ; clp->cl_cb_state =NFSD4_CB_DOWN ; -warn_no_callback_path (clp, reason); +trace_nfsd_cb_state (clp); }static void nfsd4_mark_cb_fault (struct nfs4_client *clp,int reason)
@@ -958,17 +958,20 @@ static void nfsd4_mark_cb_fault(struct nfs4_client *clp, int reason)if (test_bit (NFSD4_CLIENT_CB_UPDATE , &clp->cl_flags))return ; clp->cl_cb_state =NFSD4_CB_FAULT ; -warn_no_callback_path (clp, reason); +trace_nfsd_cb_state (clp); }static void nfsd4_cb_probe_done (struct rpc_task *task,void *calldata) { struct nfs4_client *clp = container_of(calldata, struct nfs4_client, cl_cb_null); +trace_nfsd_cb_done (clp, task->tk_status);if (task->tk_status)nfsd4_mark_cb_down (clp, task->tk_status); -else +else { clp->cl_cb_state =NFSD4_CB_UP ; +trace_nfsd_cb_state (clp); + } }static void nfsd4_cb_probe_release (void *calldata)
@@ -993,6 +996,7 @@ static const struct rpc_call_ops nfsd4_cb_probe_ops = {void nfsd4_probe_callback (struct nfs4_client *clp) { clp->cl_cb_state =NFSD4_CB_UNKNOWN ; +trace_nfsd_cb_state (clp);set_bit (NFSD4_CLIENT_CB_UPDATE , &clp->cl_flags);nfsd4_run_cb (&clp->cl_cb_null); }
@@ -1009,6 +1013,7 @@ void nfsd4_change_callback(struct nfs4_client *clp, struct nfs4_cb_conn *conn)spin_lock (&clp->cl_lock);memcpy (&clp->cl_cb_conn, conn,sizeof (struct nfs4_cb_conn ));spin_unlock (&clp->cl_lock); +trace_nfsd_cb_state (clp); }/*
@@ -1165,8 +1170,7 @@ static void nfsd4_cb_done(struct rpc_task *task, void *calldata)struct nfsd4_callback *cb = calldata;struct nfs4_client *clp = cb->cb_clp; -dprintk ("%s: minorversion=%d\n" , __func__, - clp->cl_minorversion); +trace_nfsd_cb_done (clp, task->tk_status);if (!nfsd4_cb_sequence_done (task, cb))return ;
@@ -1271,6 +1275,7 @@ static void nfsd4_process_cb_update(struct nfsd4_callback *cb)* kill the old client: */ if (clp->cl_cb_client) { +trace_nfsd_cb_shutdown (clp);rpc_shutdown_client (clp->cl_cb_client); clp->cl_cb_client =NULL ;put_cred (clp->cl_cb_cred);
@@ -1314,6 +1319,8 @@ nfsd4_run_cb_work(struct work_struct *work)struct rpc_clnt *clnt;int flags; +trace_nfsd_cb_work (clp, cb->cb_msg.rpc_proc->p_name); +if (cb->cb_need_restart) { cb->cb_need_restart = false; }else {
fs/nfsd/nfs4state.c +2 -4
@@ -2842,14 +2842,12 @@ gen_callback(struct nfs4_client *clp, struct nfsd4_setclientid *se, struct svc_r conn->cb_prog = se->se_callback_prog; conn->cb_ident = se->se_callback_ident;memcpy (&conn->cb_saddr, &rqstp->rq_daddr, rqstp->rq_daddrlen); +trace_nfsd_cb_args (clp, conn);return ; out_err: conn->cb_addr.ss_family =AF_UNSPEC ; conn->cb_addrlen =0 ; -dprintk ("NFSD: this client (clientid %08x/%08x) " -"will not receive delegations\n" , - clp->cl_clientid.cl_boot, clp->cl_clientid.cl_id); - +trace_nfsd_cb_nodelegs (clp);return ; }
fs/nfsd/trace.h +153 -0
159 lines not on this page · raw message
PATCH v4 31/33 SUNRPC: Clean up request deferral tracepoints
In reply to Chuck Lever- Rename these so they are easy to enable and search for as a set
- Move the tracepoints to get a more accurate sense of control flow
- Tracepoints should not fire on xprt shutdown
- Display memory address in case data structure had been corrupted
- Abandon dprintk in these paths
I haven't ever gotten one of these tracepoints to trigger. I wonder if we should simply remove them.
--- include/trace/events/sunrpc.h | 11 ++++++++--- net/sunrpc/svc_xprt.c | 12 ++++++------ 2 files changed, 14 insertions(+), 9 deletions(-)
include/trace/events/sunrpc.h +8 -3
@@ -1425,27 +1425,32 @@ DECLARE_EVENT_CLASS(svc_deferred_event,TP_ARGS (dr), TP_STRUCT__entry( + __field(const void *, dr) __field(u32, xid) __string(addr, dr->xprt->xpt_remotebuf) ),TP_fast_assign ( + __entry->dr = dr; __entry->xid =be32_to_cpu (*(__be32 *)(dr->args + (dr->xprt_hlen>>2 )));__assign_str (addr, dr->xprt->xpt_remotebuf); ), -TP_printk ("addr=%s xid=0x%08x" ,__get_str (addr), __entry->xid) +TP_printk ("addr=%s dr=%p xid=0x%08x" ,__get_str (addr), __entry->dr, + __entry->xid) ); +#define DEFINE_SVC_DEFERRED_EVENT (name) \ - DEFINE_EVENT(svc_deferred_event, svc_##name##_deferred, \ + DEFINE_EVENT(svc_deferred_event, svc_defer_##name, \ TP_PROTO( \ const struct svc_deferred_req *dr \ ), \ TP_ARGS(dr))DEFINE_SVC_DEFERRED_EVENT (drop); -DEFINE_SVC_DEFERRED_EVENT (revisit); +DEFINE_SVC_DEFERRED_EVENT (queue); +DEFINE_SVC_DEFERRED_EVENT (recv); TRACE_EVENT(svcsock_new_socket, TP_PROTO(
net/sunrpc/svc_xprt.c +6 -6
@@ -1146,16 +1146,15 @@ static void svc_revisit(struct cache_deferred_req *dreq, int too_many)set_bit (XPT_DEFERRED , &xprt->xpt_flags);if (too_many ||test_bit (XPT_DEAD , &xprt->xpt_flags)) {spin_unlock (&xprt->xpt_lock); -dprintk ("revisit canceled\n" ); +trace_svc_defer_drop (dr);svc_xprt_put (xprt); -trace_svc_drop_deferred (dr);kfree (dr);return ; } -dprintk ("revisit queued\n" ); dr->xprt =NULL ;list_add (&dr->handle.recent, &xprt->xpt_deferred);spin_unlock (&xprt->xpt_lock); +trace_svc_defer_queue (dr);svc_xprt_enqueue (xprt);svc_xprt_put (xprt); }
@@ -1201,22 +1200,24 @@ static struct cache_deferred_req *svc_defer(struct cache_req *req)memcpy (dr->args, rqstp->rq_arg.head[0 ].iov_base - skip, dr->argslen <<2 ); } +trace_svc_defer (rqstp);svc_xprt_get (rqstp->rq_xprt); dr->xprt = rqstp->rq_xprt;set_bit (RQ_DROPME , &rqstp->rq_flags); dr->handle.revisit = svc_revisit; -trace_svc_defer (rqstp);return &dr->handle; }/* * recv data from a deferred request into an active one */ -static int svc_deferred_recv (struct svc_rqst *rqstp) +static noinline int svc_deferred_recv(struct svc_rqst *rqstp) {struct svc_deferred_req *dr = rqstp->rq_deferred; +trace_svc_defer_recv (dr); +/* setup iov_base past transport header */ rqstp->rq_arg.head[0 ].iov_base = dr->args + (dr->xprt_hlen>>2 );/* The iov_len does not include the transport header bytes */
@@ -1247,7 +1248,6 @@ static struct svc_deferred_req *svc_deferred_dequeue(struct svc_xprt *xprt) struct svc_deferred_req, handle.recent);list_del_init (&dr->handle.recent); -trace_svc_revisit_deferred (dr); }else clear_bit (XPT_DEFERRED , &xprt->xpt_flags);spin_unlock (&xprt->xpt_lock);
PATCH v4 32/33 NFSD: Squash an annoying compiler warning
In reply to Chuck LeverClean up: Fix gcc empty-body warning when -Wextra is used.
../fs/nfsd/nfs4state.c:3898:3: warning: suggest braces around empty body in an ‘else’ statement [-Wempty-body]
--- fs/nfsd/nfs4state.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
fs/nfsd/nfs4state.c +2 -3
@@ -3881,12 +3881,11 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, unconf =find_unconfirmed_client_by_name (&clname, nn);if (unconf)unhash_client_locked (unconf); +/* We need to handle only case 1: probable callback update */ if (conf &&same_verf (&conf->cl_verifier, &clverifier)) { -/* case 1: probable callback update */ copy_clid (new, conf);gen_confirm (new, nn); - }else /* case 4 (new client) or cases 2, 3 (client reboot): */ - ; + } new->cl_minorversion =0 ;gen_callback (new, setclid, rqstp);add_to_unconfirmed (new);
PATCH v4 33/33 NFSD: Fix improperly-formatted Doxygen comments
In reply to Chuck Leverfs/nfsd/nfsctl.c:256: warning: Function parameter or member 'file' not described in 'write_unlock_ip' fs/nfsd/nfsctl.c:256: warning: Function parameter or member 'buf' not described in 'write_unlock_ip' fs/nfsd/nfsctl.c:256: warning: Function parameter or member 'size' not described in 'write_unlock_ip' fs/nfsd/nfsctl.c:295: warning: Function parameter or member 'file' not described in 'write_unlock_fs' fs/nfsd/nfsctl.c:295: warning: Function parameter or member 'buf' not described in 'write_unlock_fs' fs/nfsd/nfsctl.c:295: warning: Function parameter or member 'size' not described in 'write_unlock_fs' fs/nfsd/nfsctl.c:352: warning: Function parameter or member 'file' not described in 'write_filehandle' fs/nfsd/nfsctl.c:352: warning: Function parameter or member 'buf' not described in 'write_filehandle' fs/nfsd/nfsctl.c:352: warning: Function parameter or member 'size' not described in 'write_filehandle' fs/nfsd/nfsctl.c:434: warning: Function parameter or member 'file' not described in 'write_threads' fs/nfsd/nfsctl.c:434: warning: Function parameter or member 'buf' not described in 'write_threads' fs/nfsd/nfsctl.c:434: warning: Function parameter or member 'size' not described in 'write_threads' fs/nfsd/nfsctl.c:478: warning: Function parameter or member 'file' not described in 'write_pool_threads' fs/nfsd/nfsctl.c:478: warning: Function parameter or member 'buf' not described in 'write_pool_threads' fs/nfsd/nfsctl.c:478: warning: Function parameter or member 'size' not described in 'write_pool_threads' fs/nfsd/nfsctl.c:697: warning: Function parameter or member 'file' not described in 'write_versions' fs/nfsd/nfsctl.c:697: warning: Function parameter or member 'buf' not described in 'write_versions' fs/nfsd/nfsctl.c:697: warning: Function parameter or member 'size' not described in 'write_versions' fs/nfsd/nfsctl.c:858: warning: Function parameter or member 'file' not described in 'write_ports' fs/nfsd/nfsctl.c:858: warning: Function parameter or member 'buf' not described in 'write_ports' fs/nfsd/nfsctl.c:858: warning: Function parameter or member 'size' not described in 'write_ports' fs/nfsd/nfsctl.c:892: warning: Function parameter or member 'file' not described in 'write_maxblksize' fs/nfsd/nfsctl.c:892: warning: Function parameter or member 'buf' not described in 'write_maxblksize' fs/nfsd/nfsctl.c:892: warning: Function parameter or member 'size' not described in 'write_maxblksize' fs/nfsd/nfsctl.c:941: warning: Function parameter or member 'file' not described in 'write_maxconn' fs/nfsd/nfsctl.c:941: warning: Function parameter or member 'buf' not described in 'write_maxconn' fs/nfsd/nfsctl.c:941: warning: Function parameter or member 'size' not described in 'write_maxconn' fs/nfsd/nfsctl.c:1023: warning: Function parameter or member 'file' not described in 'write_leasetime' fs/nfsd/nfsctl.c:1023: warning: Function parameter or member 'buf' not described in 'write_leasetime' fs/nfsd/nfsctl.c:1023: warning: Function parameter or member 'size' not described in 'write_leasetime' fs/nfsd/nfsctl.c:1039: warning: Function parameter or member 'file' not described in 'write_gracetime' fs/nfsd/nfsctl.c:1039: warning: Function parameter or member 'buf' not described in 'write_gracetime' fs/nfsd/nfsctl.c:1039: warning: Function parameter or member 'size' not described in 'write_gracetime' fs/nfsd/nfsctl.c:1094: warning: Function parameter or member 'file' not described in 'write_recoverydir' fs/nfsd/nfsctl.c:1094: warning: Function parameter or member 'buf' not described in 'write_recoverydir' fs/nfsd/nfsctl.c:1094: warning: Function parameter or member 'size' not described in 'write_recoverydir' fs/nfsd/nfsctl.c:1125: warning: Function parameter or member 'file' not described in 'write_v4_end_grace' fs/nfsd/nfsctl.c:1125: warning: Function parameter or member 'buf' not described in 'write_v4_end_grace' fs/nfsd/nfsctl.c:1125: warning: Function parameter or member 'size' not described in 'write_v4_end_grace'
fs/nfsd/nfs4proc.c:1164: warning: Function parameter or member 'nss' not described in 'nfsd4_interssc_connect' fs/nfsd/nfs4proc.c:1164: warning: Function parameter or member 'rqstp' not described in 'nfsd4_interssc_connect' fs/nfsd/nfs4proc.c:1164: warning: Function parameter or member 'mount' not described in 'nfsd4_interssc_connect' fs/nfsd/nfs4proc.c:1262: warning: Function parameter or member 'rqstp' not described in 'nfsd4_setup_inter_ssc' fs/nfsd/nfs4proc.c:1262: warning: Function parameter or member 'cstate' not described in 'nfsd4_setup_inter_ssc' fs/nfsd/nfs4proc.c:1262: warning: Function parameter or member 'copy' not described in 'nfsd4_setup_inter_ssc' fs/nfsd/nfs4proc.c:1262: warning: Function parameter or member 'mount' not described in 'nfsd4_setup_inter_ssc'
--- fs/nfsd/nfs4proc.c | 7 +++---- fs/nfsd/nfsctl.c | 26 +++++++++++++------------- 2 files changed, 16 insertions(+), 17 deletions(-)
fs/nfsd/nfs4proc.c +3 -4
@@ -1155,7 +1155,7 @@ extern void nfs_sb_deactive(struct super_block *sb);#define NFSD42_INTERSSC_MOUNTOPS "vers=4.2,addr=%s,sec=sys" -/** +/* * Support one copy source server for now. */ static __be32
@@ -1245,10 +1245,9 @@ nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)mntput (ss_mnt); } -/** -* nfsd4_setup_inter_ssc -* +/* * Verify COPY destination stateid. +* * Connect to the source server with NFSv4.1. * Create the source struct file for nfsd_copy_range. * Called with COPY cstate:
fs/nfsd/nfsctl.c +13 -13
@@ -238,7 +238,7 @@ static inline struct net *netns(struct file *file)return file_inode (file)->i_sb->s_fs_info; } -/** +/* * write_unlock_ip - Release all locks used by a client * * Experimental.
@@ -277,7 +277,7 @@ static ssize_t write_unlock_ip(struct file *file, char *buf, size_t size)return nlmsvc_unlock_all_by_ip (sap); } -/** +/* * write_unlock_fs - Release all locks on a local file system * * Experimental.
@@ -327,7 +327,7 @@ static ssize_t write_unlock_fs(struct file *file, char *buf, size_t size)return error; } -/** +/* * write_filehandle - Get a variable-length NFS file handle by path * * On input, the buffer contains a '\n'-terminated C string comprised of
@@ -402,7 +402,7 @@ static ssize_t write_filehandle(struct file *file, char *buf, size_t size)return mesg - buf; } -/** +/* * write_threads - Start NFSD, or report the current number of running threads * * Input:
@@ -452,7 +452,7 @@ static ssize_t write_threads(struct file *file, char *buf, size_t size)return scnprintf (buf,SIMPLE_TRANSACTION_LIMIT ,"%d\n" , rv); } -/** +/* * write_pool_threads - Set or report the current number of threads per pool * * Input:
@@ -661,7 +661,7 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)return tlen + len; } -/** +/* * write_versions - Set or report the available NFS protocol versions * * Input:
@@ -811,7 +811,7 @@ static ssize_t __write_ports(struct file *file, char *buf, size_t size,return -EINVAL ; } -/** +/* * write_ports - Pass a socket file descriptor or transport name to listen on * * Input:
@@ -867,7 +867,7 @@ static ssize_t write_ports(struct file *file, char *buf, size_t size)int nfsd_max_blksize; -/** +/* * write_maxblksize - Set or report the current NFS blksize * * Input:
@@ -917,7 +917,7 @@ static ssize_t write_maxblksize(struct file *file, char *buf, size_t size) nfsd_max_blksize); } -/** +/* * write_maxconn - Set or report the current max number of connections * * Input:
@@ -998,7 +998,7 @@ static ssize_t nfsd4_write_time(struct file *file, char *buf, size_t size,return rv; } -/** +/* * write_leasetime - Set or report the current NFSv4 lease time * * Input:
@@ -1025,7 +1025,7 @@ static ssize_t write_leasetime(struct file *file, char *buf, size_t size)return nfsd4_write_time (file, buf, size, &nn->nfsd4_lease, nn); } -/** +/* * write_gracetime - Set or report current NFSv4 grace period time * * As above, but sets the time of the NFSv4 grace period.
@@ -1069,7 +1069,7 @@ static ssize_t __write_recoverydir(struct file *file, char *buf, size_t size, nfs4_recoverydir()); } -/** +/* * write_recoverydir - Set or report the pathname of the recovery directory * * Input:
@@ -1101,7 +1101,7 @@ static ssize_t write_recoverydir(struct file *file, char *buf, size_t size)return rv; } -/** +/* * write_v4_end_grace - release grace period for nfsd's v4.x lock manager * * Input:
Quoted 10 lines · Chuck Lever
Hi Bruce-
To address merge conflicts with Anna's tree, I've rebased this series on v5.7-rc6 plus ("SUNRPC: Split the
xdr_bufevent class"). Only two commits were changed by this rebase:SUNRPC: Move
xpt_mutexinto socketxpo_sendtomethods
SUNRPC: Add more svcsock tracepointsFeel free to make use of this version, or ignore it. :-)
Linus generally seems to prefer handling minor conflict resolutions himself over maintainers rebasing.
--b.
Quoted 97 lines
Available to view:
https://git.linux-nfs.org/?p=cel/cel-2.6.git;a=shortlog;h=refs/heads/nfsd-5.8Pull from this topic branch:
git://git.linux-nfs.org/projects/cel/cel-2.6.gitnfsd-5.8Highlights of this series:
* Remove serialization of sending RPC/RDMA Replies
* Convert the TCP socket send path to usexdr_buf::bvecs(pre-requisite for RPC-on-TLS)
* Fix svcrdma backchannel sendto return code
* Convert a number of dprintk call sites to use tracepoints
* Fix the "suggest braces around empty body in an 'else' statement" warningChanges since v3:
* Rebased on v5.7-rc6 + ("SUNRPC: Split thexdr_bufevent class")Changes since v2:
* Rebased on v5.7-rc6
* Fixed a logic error that leftXPT_DATAunset on return fromsvc_tcp_recvfrom()
* Broke down "SUNRPC: Refactorsvc_recvfrom()" to separate clean ups from logic changes
* Some superfluous clean-ups have been redacted
* Add separate tracepoints for error cases (eg,tcp_recvandtcp_recv_err)Changes since v1:
* Rebased on v5.7-rc5+
* Re-organized the series so changes interesting to linux-rdma appear together
* Addressed sparse warnings found by the kbuild test robot
* Included an additional minor clean-up: removal of the unusedSVCRDMA_DEBUGmacro
* Clarified several patch descriptions---
Chuck Lever (33):
SUNRPC: Split thexdr_bufevent class
SUNRPC: Movexpt_mutexinto socketxpo_sendtomethods
svcrdma: Clean up the tracing forrw_ctx_initerrors
svcrdma: Clean up handling ofget_rw_ctxerrors
svcrdma: Trace page overruns when constructing RDMA Reads
svcrdma: trace undersized Write chunks
svcrdma: Fix backchannel return code
svcrdma: Remove backchannel dprintk call sites
svcrdma: Rename tracepoints that record header decoding errors
svcrdma: Remove theSVCRDMA_DEBUGmacro
svcrdma: Displayed remote IP address should match stored address
svcrdma: Add tracepoints to report ->xpo_accept failures
SUNRPC: Remove kernel memory address fromsvc_xprttracepoints
SUNRPC: Tracepoint to record errors insvc_xpo_create()
SUNRPC: Trace a few more genericsvc_xprtevents
SUNRPC: Remove "#include <trace/events/skb.h>"
SUNRPC: Add more svcsock tracepoints
SUNRPC: Replace dprintk call sites in TCP state change callouts
SUNRPC: Trace server-side rpcbind registration events
SUNRPC: Renamesvc_sock::sk_reclen
SUNRPC: Restructuresvc_tcp_recv_record()
SUNRPC: Replacedprintk()call sites in TCP receive path
SUNRPC: Refactor recvfrom path dealing with incomplete TCP receives
SUNRPC: Clean upsvc_release_skb()functions
SUNRPC: Refactorsvc_recvfrom()
SUNRPC: Restructuresvc_udp_recvfrom()
SUNRPC:svc_show_status()macro should have enum definitions
NFSD: Add tracepoints to NFSD's duplicate reply cache
NFSD: Add tracepoints to the NFSD state management code
NFSD: Add tracepoints for monitoring NFSD callbacks
SUNRPC: Clean up request deferral tracepoints
NFSD: Squash an annoying compiler warning
NFSD: Fix improperly-formatted Doxygen commentsfs/nfsd/nfs4callback.c | 37 +- fs/nfsd/nfs4proc.c | 7 +- fs/nfsd/nfs4state.c | 63 ++-- fs/nfsd/nfscache.c | 57 +-- fs/nfsd/nfsctl.c | 26 +- fs/nfsd/state.h | 7 - fs/nfsd/trace.h | 345 +++++++++++++++++ include/linux/sunrpc/svc.h | 1 + include/linux/sunrpc/svc_rdma.h | 6 +- include/linux/sunrpc/svc_xprt.h | 6 + include/linux/sunrpc/svcsock.h | 6 +- include/trace/events/rpcrdma.h | 142 +++++-- include/trace/events/sunrpc.h | 419 +++++++++++++++++++-- net/sunrpc/svc.c | 19 +- net/sunrpc/svc_xprt.c | 52 +-- net/sunrpc/svcsock.c | 407 ++++++++++---------- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 121 ++---- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 21 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 92 ++--- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 10 +- net/sunrpc/xprtrdma/svc_rdma_transport.c | 55 ++- net/sunrpc/xprtsock.c | 12 +- 22 files changed, 1321 insertions(+), 590 deletions(-)--
Chuck Lever