Doorbell drop Avoidance Bug fix for iw_cxgb4

Hi All,

This patch series provides fixes for Chelsio T4/T5 adapters related to DB Drop avoidance and other small fix related to keepalive on iw-cxgb4.

The patches series is created against David Miller's 'net-next' tree. And includes patches on cxgb4 and iw_cxgb4 driver.

We would like to request this patch series to get merged via David Miller's 'net-next' tree.

We have included all the maintainers of respective drivers. Kindly review the change and let us know in case of any review comments.

Thanks

Steve Wise (2):
cxgb4/iw_cxgb4: Treat CPL_ERR_KEEPALV_NEG_ADVICE as negative advice cxgb4/iw_cxgb4: Doorbell Drop Avoidance Bug Fixes

 drivers/infiniband/hw/cxgb4/cm.c                |  24 ++--
 drivers/infiniband/hw/cxgb4/device.c            | 177 ++++++++++++++----------
 drivers/infiniband/hw/cxgb4/iw_cxgb4.h          |   9 +-
 drivers/infiniband/hw/cxgb4/provider.c          |  43 +++++-
 drivers/infiniband/hw/cxgb4/qp.c                | 140 +++++++++----------
 drivers/infiniband/hw/cxgb4/t4.h                |   6 +
 drivers/infiniband/hw/cxgb4/user.h              |   5 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4.h      |   1 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c |  87 +++++++-----
 drivers/net/ethernet/chelsio/cxgb4/sge.c        |   8 +-
 drivers/net/ethernet/chelsio/cxgb4/t4_msg.h     |   1 +
 11 files changed, 299 insertions(+), 202 deletions(-)

1.8.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

From: Steve Wise <[email protected]>

Based on original work by Anand Priyadarshee <[email protected]>.

---
 drivers/infiniband/hw/cxgb4/cm.c            | 24 ++++++++++++------------
 drivers/net/ethernet/chelsio/cxgb4/t4_msg.h |  1 +
 2 files changed, 13 insertions(+), 12 deletions(-)

index d286bde..7e98a58 100644
@@ -1647,6 +1647,15 @@ static inline int act_open_has_tid(int status)
 	       status != CPL_ERR_ARP_MISS;
 }
 
+/* Returns whether a CPL status conveys negative advice.
+ */
+static int is_neg_adv(unsigned int status)
+{
+	return status == CPL_ERR_RTX_NEG_ADVICE ||
+	       status == CPL_ERR_PERSIST_NEG_ADVICE ||
+	       status == CPL_ERR_KEEPALV_NEG_ADVICE;
+}
+
 #define ACT_OPEN_RETRY_COUNT 2
 
 static int import_ep(struct c4iw_ep *ep, int iptype, __u8 *peer_ip,

@@ -1835,7 +1844,7 @@ static int act_open_rpl(struct c4iw_dev *dev, struct sk_buff *skb)
 	PDBG("%s ep %p atid %u status %u errno %d\n", __func__, ep, atid,
 	     status, status2errno(status));
 
-	if (status == CPL_ERR_RTX_NEG_ADVICE) {
+	if (is_neg_adv(status)) {
 		printk(KERN_WARNING MOD "Connection problems for atid %u\n",
 			atid);
 		return 0;

@@ -2265,15 +2274,6 @@ static int peer_close(struct c4iw_dev *dev, struct sk_buff *skb)
 	return 0;
 }
 
-/*
- * Returns whether an ABORT_REQ_RSS message is a negative advice.
- */
-static int is_neg_adv_abort(unsigned int status)
-{
-	return status == CPL_ERR_RTX_NEG_ADVICE ||
-	       status == CPL_ERR_PERSIST_NEG_ADVICE;
-}
-
 static int peer_abort(struct c4iw_dev *dev, struct sk_buff *skb)
 {
 	struct cpl_abort_req_rss *req = cplhdr(skb);

@@ -2287,7 +2287,7 @@ static int peer_abort(struct c4iw_dev *dev, struct sk_buff *skb)
 	unsigned int tid = GET_TID(req);
 
 	ep = lookup_tid(t, tid);
-	if (is_neg_adv_abort(req->status)) {
+	if (is_neg_adv(req->status)) {
 		PDBG("%s neg_adv_abort ep %p tid %u\n", __func__, ep,
 		     ep->hwtid);
 		return 0;

@@ -3570,7 +3570,7 @@ static int peer_abort_intr(struct c4iw_dev *dev, struct sk_buff *skb)
 		kfree_skb(skb);
 		return 0;
 	}
-	if (is_neg_adv_abort(req->status)) {
+	if (is_neg_adv(req->status)) {
 		PDBG("%s neg_adv_abort ep %p tid %u\n", __func__, ep,
 		     ep->hwtid);
 		kfree_skb(skb);

index cd6874b..f2738c7 100644
@@ -116,6 +116,7 @@ enum CPL_error {
 	CPL_ERR_KEEPALIVE_TIMEDOUT = 34,
 	CPL_ERR_RTX_NEG_ADVICE     = 35,
 	CPL_ERR_PERSIST_NEG_ADVICE = 36,
+	CPL_ERR_KEEPALV_NEG_ADVICE = 37,
 	CPL_ERR_ABORT_FAILED       = 42,
 	CPL_ERR_IWARP_FLM          = 50,
 };

1.8.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

From: Hariprasad Shenai <[email protected]>
Date: Fri, 14 Mar 2014 21:52:06 +0530

Series applied, thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Steve Wise <[email protected]>

The current logic suffers from a slow response time to disable user DB usage, and also fails to avoid DB FIFO drops under heavy load. This commit fixes these deficiencies and makes the avoidance logic more optimal. This is done by more efficiently notifying the ULDs of potential DB problems, and implements a smoother flow control algorithm in iw_cxgb4, which is the ULD that puts the most load on the DB fifo.

Design:

cxgb4:

Direct ULD callback from the DB FULL/DROP interrupt handler. This allows the ULD to stop doing user DB writes as quickly as possible.

While user DB usage is disabled, the LLD will accumulate DB write events for its queues. Then once DB usage is reenabled, a single DB write is done for each queue with its accumulated write count. This reduces the load put on the DB fifo when reenabling.

iw_cxgb4:

Instead of marking each qp to indicate DB writes are disabled, we create a device-global status page that each user process maps. This allows iw_cxgb4 to only set this single bit to disable all DB writes for all user QPs vs traversing the idr of all the active QPs. If the libcxgb4 doesn't support this, then we fall back to the old approach of marking each QP. Thus we allow the new driver to work with an older libcxgb4.

When the LLD upcalls iw_cxgb4 indicating DB FULL, we disable all DB writes via the status page and transition the DB state to STOPPED. As user processes see that DB writes are disabled, they call into iw_cxgb4 to submit their DB write events. Since the DB state is in STOPPED, the QP trying to write gets enqueued on a new DB "flow control" list. As subsequent DB writes are submitted for this flow controlled QP, the amount of writes are accumulated for each QP on the flow control list. So all the user QPs that are actively ringing the DB get put on this list and the number of writes they request are accumulated.

When the LLD upcalls iw_cxgb4 indicating DB EMPTY, which is in a workq context, we change the DB state to FLOW_CONTROL, and begin resuming all the QPs that are on the flow control list. This logic runs on until the flow control list is empty or we exit FLOW_CONTROL mode (due to a DB DROP upcall, for example). QPs are removed from this list, and their accumulated DB write counts written to the DB FIFO. Sets of QPs, called chunks in the code, are removed at one time. The chunk size is 64. So 64 QPs are resumed at a time, and before the next chunk is resumed, the logic waits (blocks) for the DB FIFO to drain. This prevents resuming to quickly and overflowing the FIFO. Once the flow control list is empty, the db state transitions back to NORMAL and user QPs are again allowed to write directly to the user DB register.

The algorithm is designed such that if the DB write load is high enough, then all the DB writes get submitted by the kernel using this flow controlled approach to avoid DB drops. As the load lightens though, we resume to normal DB writes directly by user applications.

---
 drivers/infiniband/hw/cxgb4/device.c            | 177 ++++++++++++++----------
 drivers/infiniband/hw/cxgb4/iw_cxgb4.h          |   9 +-
 drivers/infiniband/hw/cxgb4/provider.c          |  43 +++++-
 drivers/infiniband/hw/cxgb4/qp.c                | 140 +++++++++----------
 drivers/infiniband/hw/cxgb4/t4.h                |   6 +
 drivers/infiniband/hw/cxgb4/user.h              |   5 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4.h      |   1 +
 drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c |  87 +++++++-----
 drivers/net/ethernet/chelsio/cxgb4/sge.c        |   8 +-
 9 files changed, 286 insertions(+), 190 deletions(-)

index 4a03385..ba7335f 100644
@@ -64,6 +64,10 @@ struct uld_ctx {
 static LIST_HEAD(uld_ctx_list);
 static DEFINE_MUTEX(dev_mutex);
 
+#define DB_FC_RESUME_SIZE 64
+#define DB_FC_RESUME_DELAY 1
+#define DB_FC_DRAIN_THRESH 0
+
 static struct dentry *c4iw_debugfs_root;
 
 struct c4iw_debugfs_data {

@@ -282,7 +286,7 @@ static const struct file_operations stag_debugfs_fops = {
 	.llseek  = default_llseek,
 };
 
-static char *db_state_str[] = {"NORMAL", "FLOW_CONTROL", "RECOVERY"};
+static char *db_state_str[] = {"NORMAL", "FLOW_CONTROL", "RECOVERY", "STOPPED"};
 
 static int stats_show(struct seq_file *seq, void *v)
 {

@@ -311,9 +315,10 @@ static int stats_show(struct seq_file *seq, void *v)
 	seq_printf(seq, "  DB FULL: %10llu\n", dev->rdev.stats.db_full);
 	seq_printf(seq, " DB EMPTY: %10llu\n", dev->rdev.stats.db_empty);
 	seq_printf(seq, "  DB DROP: %10llu\n", dev->rdev.stats.db_drop);
-	seq_printf(seq, " DB State: %s Transitions %llu\n",
+	seq_printf(seq, " DB State: %s Transitions %llu FC Interruptions %llu\n",
 		   db_state_str[dev->db_state],
-		   dev->rdev.stats.db_state_transitions);
+		   dev->rdev.stats.db_state_transitions,
+		   dev->rdev.stats.db_fc_interruptions);
 	seq_printf(seq, "TCAM_FULL: %10llu\n", dev->rdev.stats.tcam_full);
 	seq_printf(seq, "ACT_OFLD_CONN_FAILS: %10llu\n",
 		   dev->rdev.stats.act_ofld_conn_fails);

@@ -643,6 +648,12 @@ static int c4iw_rdev_open(struct c4iw_rdev *rdev)
 		printk(KERN_ERR MOD "error %d initializing ocqp pool\n", err);
 		goto err4;
 	}
+	rdev->status_page = (struct t4_dev_status_page *)
+			    __get_free_page(GFP_KERNEL);
+	if (!rdev->status_page) {
+		pr_err(MOD "error allocating status page\n");
+		goto err4;
+	}
 	return 0;
 err4:
 	c4iw_rqtpool_destroy(rdev);

@@ -656,6 +667,7 @@ err1:
 
 static void c4iw_rdev_close(struct c4iw_rdev *rdev)
 {
+	free_page((unsigned long)rdev->status_page);
 	c4iw_pblpool_destroy(rdev);
 	c4iw_rqtpool_destroy(rdev);
 	c4iw_destroy_resource(&rdev->resource);

@@ -703,18 +715,6 @@ static struct c4iw_dev *c4iw_alloc(const struct cxgb4_lld_info *infop)
 		pr_info("%s: On-Chip Queues not supported on this device.\n",
 			pci_name(infop->pdev));
 
-	if (!is_t4(infop->adapter_type)) {
-		if (!allow_db_fc_on_t5) {
-			db_fc_threshold = 100000;
-			pr_info("DB Flow Control Disabled.\n");
-		}
-
-		if (!allow_db_coalescing_on_t5) {
-			db_coalescing_threshold = -1;
-			pr_info("DB Coalescing Disabled.\n");
-		}
-	}
-
 	devp = (struct c4iw_dev *)ib_alloc_device(sizeof(*devp));
 	if (!devp) {
 		printk(KERN_ERR MOD "Cannot allocate ib device\n");

@@ -749,6 +749,7 @@ static struct c4iw_dev *c4iw_alloc(const struct cxgb4_lld_info *infop)
 	spin_lock_init(&devp->lock);
 	mutex_init(&devp->rdev.stats.lock);
 	mutex_init(&devp->db_mutex);
+	INIT_LIST_HEAD(&devp->db_fc_list);
 
 	if (c4iw_debugfs_root) {
 		devp->debugfs_root = debugfs_create_dir(

@@ -977,13 +978,16 @@ static int disable_qp_db(int id, void *p, void *data)
 
 static void stop_queues(struct uld_ctx *ctx)
 {
-	spin_lock_irq(&ctx->dev->lock);
-	if (ctx->dev->db_state == NORMAL) {
-		ctx->dev->rdev.stats.db_state_transitions++;
-		ctx->dev->db_state = FLOW_CONTROL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&ctx->dev->lock, flags);
+	ctx->dev->rdev.stats.db_state_transitions++;
+	ctx->dev->db_state = STOPPED;
+	if (ctx->dev->rdev.flags & T4_STATUS_PAGE_DISABLED)
 		idr_for_each(&ctx->dev->qpidr, disable_qp_db, NULL);
-	}
-	spin_unlock_irq(&ctx->dev->lock);
+	else
+		ctx->dev->rdev.status_page->db_off = 1;
+	spin_unlock_irqrestore(&ctx->dev->lock, flags);
 }
 
 static int enable_qp_db(int id, void *p, void *data)

@@ -994,15 +998,70 @@ static int enable_qp_db(int id, void *p, void *data)
 	return 0;
 }
 
+static void resume_rc_qp(struct c4iw_qp *qp)
+{
+	spin_lock(&qp->lock);
+	t4_ring_sq_db(&qp->wq, qp->wq.sq.wq_pidx_inc);
+	qp->wq.sq.wq_pidx_inc = 0;
+	t4_ring_rq_db(&qp->wq, qp->wq.rq.wq_pidx_inc);
+	qp->wq.rq.wq_pidx_inc = 0;
+	spin_unlock(&qp->lock);
+}
+
+static void resume_a_chunk(struct uld_ctx *ctx)
+{
+	int i;
+	struct c4iw_qp *qp;
+
+	for (i = 0; i < DB_FC_RESUME_SIZE; i++) {
+		qp = list_first_entry(&ctx->dev->db_fc_list, struct c4iw_qp,
+				      db_fc_entry);
+		list_del_init(&qp->db_fc_entry);
+		resume_rc_qp(qp);
+		if (list_empty(&ctx->dev->db_fc_list))
+			break;
+	}
+}
+
 static void resume_queues(struct uld_ctx *ctx)
 {
 	spin_lock_irq(&ctx->dev->lock);
-	if (ctx->dev->qpcnt <= db_fc_threshold &&
-	    ctx->dev->db_state == FLOW_CONTROL) {
-		ctx->dev->db_state = NORMAL;
-		ctx->dev->rdev.stats.db_state_transitions++;
-		idr_for_each(&ctx->dev->qpidr, enable_qp_db, NULL);
+	if (ctx->dev->db_state != STOPPED)
+		goto out;
+	ctx->dev->db_state = FLOW_CONTROL;
+	while (1) {
+		if (list_empty(&ctx->dev->db_fc_list)) {
+			WARN_ON(ctx->dev->db_state != FLOW_CONTROL);
+			ctx->dev->db_state = NORMAL;
+			ctx->dev->rdev.stats.db_state_transitions++;
+			if (ctx->dev->rdev.flags & T4_STATUS_PAGE_DISABLED) {
+				idr_for_each(&ctx->dev->qpidr, enable_qp_db,
+					     NULL);
+			} else {
+				ctx->dev->rdev.status_page->db_off = 0;
+			}
+			break;
+		} else {
+			if (cxgb4_dbfifo_count(ctx->dev->rdev.lldi.ports[0], 1)
+			    < (ctx->dev->rdev.lldi.dbfifo_int_thresh <<
+			       DB_FC_DRAIN_THRESH)) {
+				resume_a_chunk(ctx);
+			}
+			if (!list_empty(&ctx->dev->db_fc_list)) {
+				spin_unlock_irq(&ctx->dev->lock);
+				if (DB_FC_RESUME_DELAY) {
+					set_current_state(TASK_UNINTERRUPTIBLE);
+					schedule_timeout(DB_FC_RESUME_DELAY);
+				}
+				spin_lock_irq(&ctx->dev->lock);
+				if (ctx->dev->db_state != FLOW_CONTROL)
+					break;
+			}
+		}
 	}
+out:
+	if (ctx->dev->db_state != NORMAL)
+		ctx->dev->rdev.stats.db_fc_interruptions++;
 	spin_unlock_irq(&ctx->dev->lock);
 }
 

@@ -1028,12 +1087,12 @@ static int count_qps(int id, void *p, void *data)
 	return 0;
 }
 
-static void deref_qps(struct qp_list qp_list)
+static void deref_qps(struct qp_list *qp_list)
 {
 	int idx;
 
-	for (idx = 0; idx < qp_list.idx; idx++)
-		c4iw_qp_rem_ref(&qp_list.qps[idx]->ibqp);
+	for (idx = 0; idx < qp_list->idx; idx++)
+		c4iw_qp_rem_ref(&qp_list->qps[idx]->ibqp);
 }
 
 static void recover_lost_dbs(struct uld_ctx *ctx, struct qp_list *qp_list)

@@ -1044,17 +1103,22 @@ static void recover_lost_dbs(struct uld_ctx *ctx, struct qp_list *qp_list)
 	for (idx = 0; idx < qp_list->idx; idx++) {
 		struct c4iw_qp *qp = qp_list->qps[idx];
 
+		spin_lock_irq(&qp->rhp->lock);
+		spin_lock(&qp->lock);
 		ret = cxgb4_sync_txq_pidx(qp->rhp->rdev.lldi.ports[0],
 					  qp->wq.sq.qid,
 					  t4_sq_host_wq_pidx(&qp->wq),
 					  t4_sq_wq_size(&qp->wq));
 		if (ret) {
-			printk(KERN_ERR MOD "%s: Fatal error - "
+			pr_err(KERN_ERR MOD "%s: Fatal error - "
 			       "DB overflow recovery failed - "
 			       "error syncing SQ qid %u\n",
 			       pci_name(ctx->lldi.pdev), qp->wq.sq.qid);
+			spin_unlock(&qp->lock);
+			spin_unlock_irq(&qp->rhp->lock);
 			return;
 		}
+		qp->wq.sq.wq_pidx_inc = 0;
 
 		ret = cxgb4_sync_txq_pidx(qp->rhp->rdev.lldi.ports[0],
 					  qp->wq.rq.qid,

@@ -1062,12 +1126,17 @@ static void recover_lost_dbs(struct uld_ctx *ctx, struct qp_list *qp_list)
 					  t4_rq_wq_size(&qp->wq));
 
 		if (ret) {
-			printk(KERN_ERR MOD "%s: Fatal error - "
+			pr_err(KERN_ERR MOD "%s: Fatal error - "
 			       "DB overflow recovery failed - "
 			       "error syncing RQ qid %u\n",
 			       pci_name(ctx->lldi.pdev), qp->wq.rq.qid);
+			spin_unlock(&qp->lock);
+			spin_unlock_irq(&qp->rhp->lock);
 			return;
 		}
+		qp->wq.rq.wq_pidx_inc = 0;
+		spin_unlock(&qp->lock);
+		spin_unlock_irq(&qp->rhp->lock);
 
 		/* Wait for the dbfifo to drain */
 		while (cxgb4_dbfifo_count(qp->rhp->rdev.lldi.ports[0], 1) > 0) {

@@ -1083,36 +1152,22 @@ static void recover_queues(struct uld_ctx *ctx)
 	struct qp_list qp_list;
 	int ret;
 
-	/* lock out kernel db ringers */
-	mutex_lock(&ctx->dev->db_mutex);
-
-	/* put all queues in to recovery mode */
-	spin_lock_irq(&ctx->dev->lock);
-	ctx->dev->db_state = RECOVERY;
-	ctx->dev->rdev.stats.db_state_transitions++;
-	idr_for_each(&ctx->dev->qpidr, disable_qp_db, NULL);
-	spin_unlock_irq(&ctx->dev->lock);
-
 	/* slow everybody down */
 	set_current_state(TASK_UNINTERRUPTIBLE);
 	schedule_timeout(usecs_to_jiffies(1000));
 
-	/* Wait for the dbfifo to completely drain. */
-	while (cxgb4_dbfifo_count(ctx->dev->rdev.lldi.ports[0], 1) > 0) {
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(usecs_to_jiffies(10));
-	}
-
 	/* flush the SGE contexts */
 	ret = cxgb4_flush_eq_cache(ctx->dev->rdev.lldi.ports[0]);
 	if (ret) {
 		printk(KERN_ERR MOD "%s: Fatal error - DB overflow recovery failed\n",
 		       pci_name(ctx->lldi.pdev));
-		goto out;
+		return;
 	}
 
 	/* Count active queues so we can build a list of queues to recover */
 	spin_lock_irq(&ctx->dev->lock);
+	WARN_ON(ctx->dev->db_state != STOPPED);
+	ctx->dev->db_state = RECOVERY;
 	idr_for_each(&ctx->dev->qpidr, count_qps, &count);
 
 	qp_list.qps = kzalloc(count * sizeof *qp_list.qps, GFP_ATOMIC);

@@ -1120,7 +1175,7 @@ static void recover_queues(struct uld_ctx *ctx)
 		printk(KERN_ERR MOD "%s: Fatal error - DB overflow recovery failed\n",
 		       pci_name(ctx->lldi.pdev));
 		spin_unlock_irq(&ctx->dev->lock);
-		goto out;
+		return;
 	}
 	qp_list.idx = 0;
 

@@ -1133,29 +1188,13 @@ static void recover_queues(struct uld_ctx *ctx)
 	recover_lost_dbs(ctx, &qp_list);
 
 	/* we're almost done!  deref the qps and clean up */
-	deref_qps(qp_list);
+	deref_qps(&qp_list);
 	kfree(qp_list.qps);
 
-	/* Wait for the dbfifo to completely drain again */
-	while (cxgb4_dbfifo_count(ctx->dev->rdev.lldi.ports[0], 1) > 0) {
-		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(usecs_to_jiffies(10));
-	}
-
-	/* resume the queues */
 	spin_lock_irq(&ctx->dev->lock);
-	if (ctx->dev->qpcnt > db_fc_threshold)
-		ctx->dev->db_state = FLOW_CONTROL;
-	else {
-		ctx->dev->db_state = NORMAL;
-		idr_for_each(&ctx->dev->qpidr, enable_qp_db, NULL);
-	}
-	ctx->dev->rdev.stats.db_state_transitions++;
+	WARN_ON(ctx->dev->db_state != RECOVERY);
+	ctx->dev->db_state = STOPPED;
 	spin_unlock_irq(&ctx->dev->lock);
-
-out:
-	/* start up kernel db ringers again */
-	mutex_unlock(&ctx->dev->db_mutex);
 }
 
 static int c4iw_uld_control(void *handle, enum cxgb4_control control, ...)

@@ -1165,9 +1204,7 @@ static int c4iw_uld_control(void *handle, enum cxgb4_control control, ...)
 	switch (control) {
 	case CXGB4_CONTROL_DB_FULL:
 		stop_queues(ctx);
-		mutex_lock(&ctx->dev->rdev.stats.lock);
 		ctx->dev->rdev.stats.db_full++;
-		mutex_unlock(&ctx->dev->rdev.stats.lock);
 		break;
 	case CXGB4_CONTROL_DB_EMPTY:
 		resume_queues(ctx);

index 23eaeab..eb18f9b 100644
@@ -109,6 +109,7 @@ struct c4iw_dev_ucontext {
 
 enum c4iw_rdev_flags {
 	T4_FATAL_ERROR = (1<<0),
+	T4_STATUS_PAGE_DISABLED = (1<<1),
 };
 
 struct c4iw_stat {

@@ -130,6 +131,7 @@ struct c4iw_stats {
 	u64  db_empty;
 	u64  db_drop;
 	u64  db_state_transitions;
+	u64  db_fc_interruptions;
 	u64  tcam_full;
 	u64  act_ofld_conn_fails;
 	u64  pas_ofld_conn_fails;

@@ -150,6 +152,7 @@ struct c4iw_rdev {
 	unsigned long oc_mw_pa;
 	void __iomem *oc_mw_kva;
 	struct c4iw_stats stats;
+	struct t4_dev_status_page *status_page;
 };
 
 static inline int c4iw_fatal_error(struct c4iw_rdev *rdev)

@@ -211,7 +214,8 @@ static inline int c4iw_wait_for_reply(struct c4iw_rdev *rdev,
 enum db_state {
 	NORMAL = 0,
 	FLOW_CONTROL = 1,
-	RECOVERY = 2
+	RECOVERY = 2,
+	STOPPED = 3
 };
 
 struct c4iw_dev {

@@ -225,10 +229,10 @@ struct c4iw_dev {
 	struct mutex db_mutex;
 	struct dentry *debugfs_root;
 	enum db_state db_state;
-	int qpcnt;
 	struct idr hwtid_idr;
 	struct idr atid_idr;
 	struct idr stid_idr;
+	struct list_head db_fc_list;
 };
 
 static inline struct c4iw_dev *to_c4iw_dev(struct ib_device *ibdev)

@@ -432,6 +436,7 @@ struct c4iw_qp_attributes {
 
 struct c4iw_qp {
 	struct ib_qp ibqp;
+	struct list_head db_fc_entry;
 	struct c4iw_dev *rhp;
 	struct c4iw_ep *ep;
 	struct c4iw_qp_attributes attr;

index e73ace7..eeca8b1 100644
@@ -300,6 +300,7 @@ struct t4_sq {
 	u16 cidx;
 	u16 pidx;
 	u16 wq_pidx;
+	u16 wq_pidx_inc;
 	u16 flags;
 	short flush_cidx;
 };

@@ -324,6 +325,7 @@ struct t4_rq {
 	u16 cidx;
 	u16 pidx;
 	u16 wq_pidx;
+	u16 wq_pidx_inc;
 };
 
 struct t4_wq {

@@ -609,3 +611,7 @@ static inline void t4_set_cq_in_error(struct t4_cq *cq)
 	((struct t4_status_page *)&cq->queue[cq->size])->qp_err = 1;
 }
 #endif
+
+struct t4_dev_status_page {
+	u8 db_off;
+};

index 32b754c..11ccd27 100644
@@ -70,4 +70,9 @@ struct c4iw_create_qp_resp {
 	__u32 qid_mask;
 	__u32 flags;
 };
+
+struct c4iw_alloc_ucontext_resp {
+	__u64 status_page_key;
+	__u32 status_page_size;
+};
 #endif

index 50abe1d..32db377 100644
@@ -500,6 +500,7 @@ struct sge_txq {
 	spinlock_t db_lock;
 	int db_disabled;
 	unsigned short db_pidx;
+	unsigned short db_pidx_inc;
 	u64 udb;
 };
 

index 46429f9..d4db382 100644
@@ -860,9 +860,10 @@ static void cxgb_pio_copy(u64 __iomem *dst, u64 *src)
 static inline void ring_tx_db(struct adapter *adap, struct sge_txq *q, int n)
 {
 	unsigned int *wr, index;
+	unsigned long flags;
 
 	wmb();            /* write descriptors before telling HW */
-	spin_lock(&q->db_lock);
+	spin_lock_irqsave(&q->db_lock, flags);
 	if (!q->db_disabled) {
 		if (is_t4(adap->params.chip)) {
 			t4_write_reg(adap, MYPF_REG(SGE_PF_KDOORBELL),

@@ -878,9 +879,10 @@ static inline void ring_tx_db(struct adapter *adap, struct sge_txq *q, int n)
 				writel(n,  adap->bar2 + q->udb + 8);
 			wmb();
 		}
-	}
+	} else
+		q->db_pidx_inc += n;
 	q->db_pidx = q->pidx;
-	spin_unlock(&q->db_lock);
+	spin_unlock_irqrestore(&q->db_lock, flags);
 }
 
 /**

1.8.4

[...]

[...]

[...]

Yann Droneaud
OPTEYA


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

You're correct.

Sounds good. Can you provide patches for these?

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

The c4iw_alloc_ucontext_resp struct has a 4 byte hole after the last member and we should clear it before passing it to the user.

index e36d2a2..a72aaa7 100644
@@ -107,7 +107,7 @@ static struct ib_ucontext *c4iw_alloc_ucontext(struct ib_device *ibdev,
 	struct c4iw_ucontext *context;
 	struct c4iw_dev *rhp = to_c4iw_dev(ibdev);
 	static int warned;
-	struct c4iw_alloc_ucontext_resp uresp;
+	struct c4iw_alloc_ucontext_resp uresp = {};
 	int ret = 0;
 	struct c4iw_mm_entry *mm = NULL;
 

Hi,

It's not the proper fix for this issue: an explicit padding has to be added (and initialized), see "Re: [PATCH net-next 2/2] cxgb4/iw_cxgb4:
Doorbell Drop Avoidance Bug Fixes"
http://marc.info/?i[email protected]

In its current form, the c4iw_alloc_ucontext_resp structure does not require padding on i386, so a 32bits userspace program using this structure against a x86_64 kernel will make the kernel do a buffer overflow in userspace, likely on stack, as answer of a GET_CONTEXT request:

#include <infiniband/verbs.h>
#include <infiniband/kern-abi.h>

#define IBV_INIT_CMD_RESP(cmd, size, opcode, out, outsize) \

do {                                                       \
    (cmd)->command = IB_USER_VERBS_CMD_##opcode;           \
    (cmd)->in_words  = (size) / 4;                         \
    (cmd)->out_words = (outsize) / 4;                      \
    (cmd)->response  = (out);                              \

} while (0)

struct c4iw_alloc_ucontext_resp {
        struct ibv_get_context_resp ibv_resp;
        __u64 status_page_key;
        __u32 status_page_size;
};

struct ibv_context *alloc_context(struct ibv_device *ibdev,
int cmd_fd)

{
    struct ibv_get_context cmd;
    struct c4iw_alloc_ucontext_resp resp;
    ssize_t sret;
    ...

    IBV_INIT_CMD_RESP(&cmd, sizeof(cmd),
                      GET_CONTEXT,
                      &resp, sizeof(resp));
    ...
    sret = write(context->cmd_fd,
                 &cmd,
                 sizeof(cmd));

    if (sret != sizeof(cmd)) {
        int err = errno;
        fprintf(stderr, "GET_CONTEXT failed: %d (%s)\n",
                err, strerror(err));
        ...
    }
    ...
}

Unfortunately, it's not the only structure which has this problem. I'm currently preparing a report on this issue for this driver (cxgb4) and another.

Yann Droneaud
OPTEYA

RnJvbTogWWFubiBEcm9uZWF1ZA0KPiBIaSwNCj4gDQo+IExlIHZlbmRyZWRpIDI4IG1hcnMgMjAx NCAgMTE6MjQgKzAzMDAsIERhbiBDYXJwZW50ZXIgYSBjcml0IDoNCj4gPiBUaGUgYzRpd19hbGxv Y191Y29udGV4dF9yZXNwIHN0cnVjdCBoYXMgYSA0IGJ5dGUgaG9sZSBhZnRlciB0aGUgbGFzdA0K PiA+IG1lbWJlciBhbmQgd2Ugc2hvdWxkIGNsZWFyIGl0IGJlZm9yZSBwYXNzaW5nIGl0IHRvIHRo ZSB1c2VyLg0KPiA+DQo+ID4gRml4ZXM6IDA1ZWIyMzg5M2MyYyAoJ2N4Z2I0L2l3X2N4Z2I0OiBE b29yYmVsbCBEcm9wIEF2b2lkYW5jZSBCdWcgRml4ZXMnKQ0KPiA+IFNpZ25lZC1vZmYtYnk6IERh biBDYXJwZW50ZXIgPGRhbi5jYXJwZW50ZXJAb3JhY2xlLmNvbT4NCj4gPg0KPiANCj4gSXQncyBu b3QgdGhlIHByb3BlciBmaXggZm9yIHRoaXMgaXNzdWU6IGFuIGV4cGxpY2l0IHBhZGRpbmcgaGFz IHRvIGJlDQo+IGFkZGVkIChhbmQgaW5pdGlhbGl6ZWQpLCBzZWUgIlJlOiBbUEFUQ0ggbmV0LW5l eHQgMi8yXSBjeGdiNC9pd19jeGdiNDoNCj4gRG9vcmJlbGwgRHJvcCBBdm9pZGFuY2UgQnVnIEZp eGVzIg0KPiBodHRwOi8vbWFyYy5pbmZvLz9pPTEzOTU4NDg5NzcuMzI5Ny4xNS5jYW1lbEBsb2Nh bGhvc3QubG9jYWxkb21haW4NCj4gDQo+IEluIGl0cyBjdXJyZW50IGZvcm0sIHRoZSBjNGl3X2Fs bG9jX3Vjb250ZXh0X3Jlc3Agc3RydWN0dXJlIGRvZXMgbm90DQo+IHJlcXVpcmUgcGFkZGluZyBv biBpMzg2LCBzbyBhIDMyYml0cyB1c2Vyc3BhY2UgcHJvZ3JhbSB1c2luZyB0aGlzDQo+IHN0cnVj dHVyZSBhZ2FpbnN0IGEgeDg2XzY0IGtlcm5lbCB3aWxsIG1ha2UgdGhlIGtlcm5lbCBkbyBhIGJ1 ZmZlcg0KPiBvdmVyZmxvdyBpbiB1c2Vyc3BhY2UsIGxpa2VseSBvbiBzdGFjaywgYXMgYW5zd2Vy IG9mIGEgR0VUX0NPTlRFWFQNCj4gcmVxdWVzdDoNCi4uLg0KPiBzdHJ1Y3QgYzRpd19hbGxvY191 Y29udGV4dF9yZXNwIHsNCj4gICAgICAgICBzdHJ1Y3QgaWJ2X2dldF9jb250ZXh0X3Jlc3AgaWJ2 X3Jlc3A7DQo+ICAgICAgICAgX191NjQgc3RhdHVzX3BhZ2Vfa2V5Ow0KPiAgICAgICAgIF9fdTMy IHN0YXR1c19wYWdlX3NpemU7DQo+IH07DQoNCk9yIGFkZCBfX2F0dHJpYnV0ZV9fKChhbGlnbmVk KDQpKSkgdG8gdGhlIDY0Yml0IGZpZWxkcy4NCkFuZCBtYXliZSBhIGNvbXBpbGUgdGltZSBhc3Nl cnQgb24gdGhlIGxlbmd0aCBvZiB0aGUgc3RydWN0dXJlLg0KU2luY2UgaXQgaXMgcGFydCBvZiBh biBBQkkgaXQgbXVzdCBub3QgY2hhbmdlDQoNCglEYXZpZA0KDQo

This information leak is still present in linux-next. These days we count those things as security vulnerabilities with CVEs and everything.

When is it likely to get fixed?

regards,
dan carpenter

Hi,

Please find 4 patches which fix some issues regarding missing explicit padding at end of structure exchanged between kernel and userspace.

These makes i386 userspace libraries and x86_64 kernel disagree about the size of the structures.

Additionally, as reported by Dan Carpenter, in one case, stack information can be leaked by the kernel to userspace due to implicit padding being not initialized.

Unfortunately, the data structure cannot be fixed alone as it would break existing applications. So in order to remain compatible with i386 libraries, providers (hw) functions are modified to use the input length to guess the expected format of the command in order to check the content of the reserved field for future usage. Other are modified to not write the padding field in response to make the kernel able to handle gracefully i386 userspace on x86_64.

For full coherency, patches against the userspace libraries (libcxgb4 and libmlx5) will be submitted as a followup to update the data structure on userspace side.

Yann Droneaud (4):
RDMA/mlx5: add missing padding at end of struct mlx5_ib_create_cq RDMA/mlx5: add missing padding at end of struct mlx5_ib_create_srq RDMA/cxgb4: add missing padding at end of struct c4iw_create_cq_resp RDMA/cxgb4: add missing padding at end of struct
c4iw_alloc_ucontext_resp

 drivers/infiniband/hw/cxgb4/cq.c       |  8 ++++++--
 drivers/infiniband/hw/cxgb4/provider.c |  8 ++++++--
 drivers/infiniband/hw/cxgb4/user.h     |  2 ++
 drivers/infiniband/hw/mlx5/cq.c        | 13 +++++++++++--
 drivers/infiniband/hw/mlx5/srq.c       | 18 +++++++++++++++---
 drivers/infiniband/hw/mlx5/user.h      |  2 ++
 6 files changed, 42 insertions(+), 9 deletions(-)

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

i386 ABI disagree with most other ABIs regarding alignment of data type larger than 4 bytes: on most ABIs a padding must be added at end of the structures, while it is not required on i386.

So for most ABI struct mlx5_ib_create_cq get padded to be aligned on a 8 bytes multiple, while for i386, such padding is not added.

Tool pahole could be used to find such implicit padding:

  $ pahole --anon_include \
  	 --nested_anon_include \
  	 --recursive \
  	 --class_name mlx5_ib_create_cq \
  	 drivers/infiniband/hw/mlx5/mlx5_ib.o

Then, structure layout can be compared between i386 and x86_64:

  +++ obj-i386/drivers/infiniband/hw/mlx5/mlx5_ib.o.pahole.txt    2014-03-28 11:43:07.386413682 +0100
  --- obj-x86_64/drivers/infiniband/hw/mlx5/mlx5_ib.o.pahole.txt  2014-03-27 13:06:17.788472721 +0100
  @@ -34,9 +34,8 @@ struct mlx5_ib_create_cq {
          __u64                      db_addr;              /*     8     8 */
          __u32                      cqe_size;             /*    16     4 */

  -       /* size: 20, cachelines: 1, members: 3 */
  -       /* last cacheline: 20 bytes */
  +       /* size: 24, cachelines: 1, members: 3 */
  +       /* padding: 4 */
  +       /* last cacheline: 24 bytes */

};

This ABI disagreement will make an x86_64 kernel try to read past the buffer provided by an i386 binary.

When boundary check will be implemented, a x86_64 kernel will refuse to read past the i386 userspace provided buffer and the uverb will fail.

Anyway, if the structure lay in memory on a page boundary and next page is not mapped, ib_copy_from_udata() will fail when trying to read the 4 bytes of padding and the uverb will fail.

This patch makes create_cq_user() takes care of the input data size to handle the case where no padding is provided.

This way, x86_64 kernel will be able to handle struct mlx5_ib_create_cq as sent by unpatched and patched i386 libmlx5.

---
 drivers/infiniband/hw/mlx5/cq.c   | 13 +++++++++++--
 drivers/infiniband/hw/mlx5/user.h |  1 +
 2 files changed, 12 insertions(+), 2 deletions(-)

index 62bb6b49dc1d..5996132e799d 100644
@@ -32,6 +32,7 @@
 
 #include <linux/kref.h>
 #include <rdma/ib_umem.h>
+#include <rdma/ib_user_verbs.h>
 #include "mlx5_ib.h"
 #include "user.h"
 

@@ -607,8 +608,16 @@ static int create_cq_user(struct mlx5_ib_dev *dev, struct ib_udata *udata,
 	int ncont;
 	int err;
 
-	if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd)))
-		return -EFAULT;
+	if (udata->inlen - sizeof(struct ib_uverbs_cmd_hdr) < sizeof(ucmd)) {
+		if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd) - sizeof(ucmd.reserved)))
+			return -EFAULT;
+	} else {
+		if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd)))
+			return -EFAULT;
+
+		if (ucmd.reserved != 0)
+			return -EINVAL;
+	}
 
 	if (ucmd.cqe_size != 64 && ucmd.cqe_size != 128)
 		return -EINVAL;

index 0f4f8e42a17f..d44ecd2c2faf 100644
@@ -91,6 +91,7 @@ struct mlx5_ib_create_cq {
 	__u64	buf_addr;
 	__u64	db_addr;
 	__u32	cqe_size;
+	__u32	reserved; /* explicit padding (optional on i386) */
 };
 
 struct mlx5_ib_create_cq_resp {

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

i386 ABI disagree with most other ABIs regarding alignment of data type larger than 4 bytes: on most ABIs a padding must be added at end of the structures, while it is not required on i386.

So for most ABI struct c4iw_create_cq_resp get implicitly padded to be aligned on a 8 bytes multiple, while for i386, such padding is not added.

Tool pahole could be used to find such implicit padding:

  $ pahole --anon_include \
           --nested_anon_include \
           --recursive \
           --class_name c4iw_create_cq_resp \
           drivers/infiniband/hw/cxgb4/iw_cxgb4.o

Then, structure layout can be compared between i386 and x86_64:

  +++ obj-i386/drivers/infiniband/hw/cxgb4/iw_cxgb4.o.pahole.txt   2014-03-28 11:43:05.547432195 +0100
  --- obj-x86_64/drivers/infiniband/hw/cxgb4/iw_cxgb4.o.pahole.txt 2014-03-28 10:55:10.990133017 +0100
  @@ -14,9 +13,8 @@ struct c4iw_create_cq_resp {
          __u32                      size;                 /*    28     4 */
          __u32                      qid_mask;             /*    32     4 */

  -       /* size: 36, cachelines: 1, members: 6 */
  -       /* last cacheline: 36 bytes */
  +       /* size: 40, cachelines: 1, members: 6 */
  +       /* padding: 4 */
  +       /* last cacheline: 40 bytes */

};

This ABI disagreement will make an x86_64 kernel try to write past the buffer provided by an i386 binary.

When boundary check will be implemented, the x86_64 kernel will refuse to write past the i386 userspace provided buffer and the uverbs will fail.

If the structure lay in memory on a page boundary and next page is not mapped, ib_copy_to_udata() will fail and the uverbs will fail.

This patch adds an explicit padding at end of structure c4iw_create_cq_resp, and, like 92b0ca7cb149 ('IB/mlx5: Fix stack info leak in mlx5_ib_alloc_ucontext()'), makes function c4iw_create_cq() not writting this padding field to userspace. This way, x86_64 kernel will be able to write struct c4iw_create_cq_resp as expected by unpatched and patched i386 libcxgb4.

---
 drivers/infiniband/hw/cxgb4/cq.c   | 8 ++++++--
 drivers/infiniband/hw/cxgb4/user.h | 1 +
 2 files changed, 7 insertions(+), 2 deletions(-)

index cfaa56ada189..5cb3cb4b4c25 100644
@@ -940,7 +940,6 @@ struct ib_cq *c4iw_create_cq(struct ib_device *ibdev, int entries,
 		if (!mm2)
 			goto err4;
 
-		memset(&uresp, 0, sizeof(uresp));
 		uresp.qid_mask = rhp->rdev.cqmask;
 		uresp.cqid = chp->cq.cqid;
 		uresp.size = chp->cq.size;

@@ -951,7 +950,7 @@ struct ib_cq *c4iw_create_cq(struct ib_device *ibdev, int entries,
 		uresp.gts_key = ucontext->key;
 		ucontext->key += PAGE_SIZE;
 		spin_unlock(&ucontext->mmap_lock);
-		ret = ib_copy_to_udata(udata, &uresp, sizeof uresp);
+		ret = ib_copy_to_udata(udata, &uresp, sizeof(uresp) - sizeof(uresp.reserved));
 		if (ret)
 			goto err5;
 

index 11ccd276e5d9..9b7534b5f07d 100644
@@ -48,6 +48,7 @@ struct c4iw_create_cq_resp {
 	__u32 cqid;
 	__u32 size;
 	__u32 qid_mask;
+	__u32 reserved; /* explicit padding (optional for i386) */
 };
 
 

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

i386 ABI disagree with most other ABIs regarding alignment of data type larger than 4 bytes: on most ABIs a padding must be added at end of the structures, while it is not required on i386.

So for most ABI struct c4iw_alloc_ucontext_resp get implicitly padded to be aligned on a 8 bytes multiple, while for i386, such padding is not added.

Tool pahole could be used to find such implicit padding:

  $ pahole --anon_include \
           --nested_anon_include \
           --recursive \
           --class_name c4iw_alloc_ucontext_resp \
           drivers/infiniband/hw/cxgb4/iw_cxgb4.o

Then, structure layout can be compared between i386 and x86_64:

  +++ obj-i386/drivers/infiniband/hw/cxgb4/iw_cxgb4.o.pahole.txt   2014-03-28 11:43:05.547432195 +0100
  --- obj-x86_64/drivers/infiniband/hw/cxgb4/iw_cxgb4.o.pahole.txt 2014-03-28 10:55:10.990133017 +0100
  @@ -2,9 +2,8 @@ struct c4iw_alloc_ucontext_resp {
          __u64                      status_page_key;      /*     0     8 */
          __u32                      status_page_size;     /*     8     4 */

  -       /* size: 12, cachelines: 1, members: 2 */
  -       /* last cacheline: 12 bytes */
  +       /* size: 16, cachelines: 1, members: 2 */
  +       /* padding: 4 */
  +       /* last cacheline: 16 bytes */

};

This ABI disagreement will make an x86_64 kernel try to write past the buffer provided by an i386 binary.

When boundary check will be implemented, the x86_64 kernel will refuse to write past the i386 userspace provided buffer and the uverbs will fail.

If the structure lay in memory on a page boundary and next page is not mapped, ib_copy_to_udata() will fail and the uverb will fail.

Additionally, as reported by Dan Carpenter, without the implicit padding being properly cleared, an information leak would take place in most architectures.

This patch adds an explicit padding to struct c4iw_alloc_ucontext_resp, and, like 92b0ca7cb149 ('IB/mlx5: Fix stack info leak in mlx5_ib_alloc_ucontext()'), makes function c4iw_alloc_ucontext() not writting this padding field to userspace. This way, x86_64 kernel will be able to write struct c4iw_alloc_ucontext_resp as expected by unpatched and patched i386 libcxgb4.

---
 drivers/infiniband/hw/cxgb4/provider.c | 8 ++++++--
 drivers/infiniband/hw/cxgb4/user.h     | 1 +
 2 files changed, 7 insertions(+), 2 deletions(-)

index a94a3e12c349..9ba01d524f51 100644
@@ -122,7 +122,7 @@ static struct ib_ucontext *c4iw_alloc_ucontext(struct ib_device *ibdev,
 	INIT_LIST_HEAD(&context->mmaps);
 	spin_lock_init(&context->mmap_lock);
 
-	if (udata->outlen < sizeof(uresp)) {
+	if (udata->outlen < sizeof(uresp) - sizeof(uresp.reserved)) {
 		if (!warned++)
 			pr_err(MOD "Warning - downlevel libcxgb4 (non-fatal), device status page disabled.");
 		rhp->rdev.flags |= T4_STATUS_PAGE_DISABLED;

@@ -134,13 +134,13 @@ static struct ib_ucontext *c4iw_alloc_ucontext(struct ib_device *ibdev,
 		}
 
 		uresp.status_page_size = PAGE_SIZE;
 
 		spin_lock(&context->mmap_lock);
 		uresp.status_page_key = context->key;
 		context->key += PAGE_SIZE;
 		spin_unlock(&context->mmap_lock);
 
-		ret = ib_copy_to_udata(udata, &uresp, sizeof(uresp));
+		ret = ib_copy_to_udata(udata, &uresp, sizeof(uresp) - sizeof(uresp.reserved));
 		if (ret)
 			goto err_mm;
 

index 9b7534b5f07d..cbd0ce170728 100644
@@ -75,5 +75,6 @@ struct c4iw_create_qp_resp {
 struct c4iw_alloc_ucontext_resp {
 	__u64 status_page_key;
 	__u32 status_page_size;
+	__u32 reserved; /* explicit padding (optional for i386) */
 };
 #endif

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

<snip>

Hi,

I've hand-edited it before sending, but I've tested it with git am against linux-next (as of today).

Anyway, I'm rolling an updated patchset. Sorry for the inconvenience.

Thanks.

Yann Droneaud
OPTEYA


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

No problem. I hand applied it and tested these 2 + the libcxgb4 changes and they seem to work fine.

Steve.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi,

Please find two patches for libcxgb4 to keep the library in sync regarding the kernel changes I've submitted: explicit padding must be added at end of the structure so that i386 userspace library has the same structure layout than x86_64 kernel.

See http://marc.info/?i=cover.1399216475.git.ydroneaud-RlY5vtjFyJ3QT0dZR+AlfA@public.gmane.org

Yann Droneaud (2):
kernel abi: adds explicit padding in struct c4iw_create_cq_resp kernel abi: adds explicit padding in struct c4iw_alloc_ucontext_resp

 src/cxgb4-abi.h | 2 ++
 src/dev.c       | 5 +++++
 src/verbs.c     | 5 +++++
 3 files changed, 12 insertions(+)

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

i386 ABI disagree with most other ABIs regarding alignment of data type larger than 4 bytes: on most ABIs a padding must be added at end of the structures, while it is not required on i386.

Such ABI disagreement will make an x86_64 kernel try to write past the struct c4iw_create_cq_resp buffer provided by an i386 userspace binary. As struct c4iw_create_cq_resp is likely on stack, see function c4iw_create_cq(), side effects are expected.

On kernel side, this structure was added for kernel v2.6.35-rc1 by following commit.

Commit cfdda9d764362ab77b11a410bb928400e6520d57
Author: Steve Wise <[email protected]>
Date: Wed Apr 21 15:30:06 2010 -0700

RDMA/cxgb4: Add driver for Chelsio T4 RNIC

If boundary check is implemented on kernel side, the x86_64 kernel will refuse to write past the i386 userspace provided buffer and the uverbs will fail.

To fix these issues, this patch adds an explicit padding at end of structure so that i386 and others ABI share the same structure layout. This patch makes c4iw_create_cq() check for a value in the padding field to detect newer kernel using the field for a future purpose (only activated in debug).

With this patch, libcxgb4 will work against older kernel and newer patched kernel.

---
 src/cxgb4-abi.h | 1 +
 src/verbs.c     | 5 +++++
 2 files changed, 6 insertions(+)

index d70b0f132a7f..23870f66dc0d 100644
@@ -53,6 +53,7 @@ struct c4iw_create_cq_resp {
 	__u32 cqid;
 	__u32 size;
 	__u32 qid_mask;
+	__u32 reserved;
 };
 
 enum {

index ab4a45d7cdbc..4a6c1b47bc9e 100644
@@ -181,12 +181,17 @@ struct ibv_cq *c4iw_create_cq(struct ibv_context *context, int cqe,
 		return NULL;
 	}
 
+	resp.reserved = 0;
 	ret = ibv_cmd_create_cq(context, cqe, channel, comp_vector,
 				&chp->ibv_cq, &cmd, sizeof cmd,
 				&resp.ibv_resp, sizeof resp);
 	if (ret)
 		goto err1;
 
+	if (resp.reserved)
+		PDBG("%s c4iw_create_cq_resp reserved field modified by kernel\n",
+		     __FUNCTION__);
+
 	pthread_spin_init(&chp->lock, PTHREAD_PROCESS_PRIVATE);
 #ifdef STALL_DETECTION
 	gettimeofday(&chp->time, NULL);

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

i386 ABI disagree with most other ABIs regarding alignment of data type larger than 4 bytes: on most ABIs a padding must be added at end of the structures, while it is not required on i386.

Such ABI disagreement will make an x86_64 kernel try to write past the struct c4iw_alloc_ucontext_resp buffer provided by an i386 userspace binary. As struct c4iw_alloc_ucontext_resp is likely on stack, see function c4iw_alloc_context(), side effects are expected.

On kernel side, this structure was modified for kernel v3.15-rc1 by following commit:

Commit 05eb23893c2cf9502a9cec0c32e7f1d1ed2895c8
Author: Steve Wise <[email protected]>
Date: Fri Mar 14 21:52:08 2014 +0530

cxgb4/iw_cxgb4: Doorbell Drop Avoidance Bug Fixes

If boundary check is implemented on kernel side, the x86_64 kernel will instead refuse to write past the i386 userspace provided buffer and the uverbs will fail.

To fix these issues, this patch adds an explicit padding at end of structure so that i386 and others ABI share the same structure layout. This patch makes c4iw_alloc_context() check for a value in the padding field to detect newer kernel using the field for a future purpose (only activated in debug).

With this patch, libcxgb4 will work against older kernel and newer patched kernel.

---
 src/cxgb4-abi.h | 1 +
 src/dev.c       | 5 +++++
 2 files changed, 6 insertions(+)

index 23870f66dc0d..0b9f4d99d0e7 100644
@@ -38,6 +38,7 @@ struct c4iw_alloc_ucontext_resp {
 	struct ibv_get_context_resp ibv_resp;
 	__u64 status_page_key;
 	__u32 status_page_size;
+	__u32 reserved;
 };
 
 struct c4iw_alloc_pd_resp {

index 3236e6b2db6d..f66df71105e5 100644
@@ -125,10 +125,15 @@ static struct ibv_context *c4iw_alloc_context(struct ibv_device *ibdev,
 	context->ibv_ctx.cmd_fd = cmd_fd;
 
 	resp.status_page_size = 0;
+	resp.reserved = 0;
 	if (ibv_cmd_get_context(&context->ibv_ctx, &cmd, sizeof cmd,
 				&resp.ibv_resp, sizeof resp))
 		goto err_free;
 
+	if (resp.reserved)
+		PDBG("%s c4iw_alloc_ucontext_resp reserved field modified by kernel\n",
+		     __FUNCTION__);
+
 	context->status_page_size = resp.status_page_size;
 	if (resp.status_page_size) {
 		context->status_page = mmap(NULL, resp.status_page_size,

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Applied. Thanks!

Steve.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

i386 ABI disagree with most other ABIs regarding alignment of data type larger than 4 bytes: on most ABIs a padding must be added at end of the structures, while it is not required on i386.

Such ABI disagreement will make an x86_64 kernel try to read past a buffer provided by an i386 binary, as the latter will not have the expected padding for struct mlx5_create_cq and mlx5_create_srq.

On kernel side, these structures were added for kernel v3.11-rc1 by following commit:

Commit e126ba97dba9edeb6fafa3665b5f8497fc9cdf8c
Author: Eli Cohen <eli-VPRAkNaXOzVWk0Htik3J/[email protected]>
Date: Sun Jul 7 17:25:49 2013 +0300

mlx5: Add driver for Mellanox Connect-IB adapters

If future kernel is to use the padding for extension, on a x86_64 unpatched kernel, it might read garbage as it would read past the i386 provided buffer.

In this other hand, if boundary check is implemented on kernel side, the x86_64 kernel will refuse to read past the i386 userspace provided buffer for struct mlx5_create_cq and mlx5_create_srq, making the uverbs fail.

To address all these issues, this patch add an explicit padding at end of structures and initialize it so that i386 and others ABI share the same structure layout.

With this patch, libmlx5 will run on older kernel and newer patched kernel.

---
 src/mlx5-abi.h | 2 ++
 src/verbs.c    | 6 ++++--
 2 files changed, 6 insertions(+), 2 deletions(-)

index 6f98e62c59d5..980b24910403 100644
@@ -83,6 +83,7 @@ struct mlx5_create_cq {
 	__u64				buf_addr;
 	__u64				db_addr;
 	__u32				cqe_size;
+	__u32				reserved;
 };
 
 struct mlx5_create_cq_resp {

@@ -95,6 +96,7 @@ struct mlx5_create_srq {
 	__u64				buf_addr;
 	__u64				db_addr;
 	__u32				flags;
+	__u32				reserved;
 };
 
 struct mlx5_create_srq_resp {

index 7201e94925c5..1de8692e5264 100644
@@ -261,7 +261,6 @@ struct ibv_cq *mlx5_create_cq(struct ibv_context *context, int cqe,
 		return NULL;
 	}
 
-	memset(&cmd, 0, sizeof cmd);
 	cq->cons_index = 0;
 
 	if (mlx5_spinlock_init(&cq->lock))

@@ -307,6 +306,7 @@ struct ibv_cq *mlx5_create_cq(struct ibv_context *context, int cqe,
 	cmd.buf_addr = (uintptr_t) cq->buf_a.buf;
 	cmd.db_addr  = (uintptr_t) cq->dbrec;
 	cmd.cqe_size = cqe_sz;
+	cmd.reserved = 0;
 
 	ret = ibv_cmd_create_cq(context, ncqe - 1, channel, comp_vector,
 				&cq->ibv_cq, &cmd.ibv_cmd, sizeof cmd,

@@ -442,7 +442,6 @@ struct ibv_srq *mlx5_create_srq(struct ibv_pd *pd,
 	}
 	ibsrq = &srq->srq;
 
-	memset(&cmd, 0, sizeof cmd);
 	if (mlx5_spinlock_init(&srq->lock)) {
 		fprintf(stderr, "%s-%d:\n", __func__, __LINE__);
 		goto err;

@@ -490,6 +489,9 @@ struct ibv_srq *mlx5_create_srq(struct ibv_pd *pd,
 	srq->wq_sig = srq_sig_enabled();
 	if (srq->wq_sig)
 		cmd.flags = MLX5_SRQ_FLAG_SIGNATURE;
+	else
+		cmd.flags = 0;
+	cmd.reserved = 0;
 
 	attr->attr.max_sge = srq->max_gs;
 	pthread_mutex_lock(&ctx->srq_table_mutex);

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi,

I've made a review of all others drivers. I've identified the following structures as part of ABI:

cxgb3/iw_cxgb3.o     struct iwch_create_cq_req
cxgb3/iw_cxgb3.o     struct iwch_create_cq_resp
cxgb3/iw_cxgb3.o     struct iwch_create_qp_resp
cxgb3/iw_cxgb3.o     struct iwch_reg_user_mr_resp

cxgb4/iw_cxgb4.o     struct c4iw_alloc_ucontext_resp
cxgb4/iw_cxgb4.o     struct c4iw_create_cq_resp
cxgb4/iw_cxgb4.o     struct c4iw_create_qp_resp

ehca/ib_ehca.o       struct ehca_create_cq_resp
ehca/ib_ehca.o       struct ehca_create_qp_resp
ehca/ib_ehca.o       struct ipzu_queue_resp

mlx4/mlx4_ib.o       struct mlx4_ib_alloc_ucontext_resp
mlx4/mlx4_ib.o       struct mlx4_ib_alloc_ucontext_resp_v3
mlx4/mlx4_ib.o       struct mlx4_ib_create_cq
mlx4/mlx4_ib.o       struct mlx4_ib_create_qp
mlx4/mlx4_ib.o       struct mlx4_ib_create_srq
mlx4/mlx4_ib.o       struct mlx4_ib_resize_cq

mlx5/mlx5_ib.o       struct mlx5_ib_alloc_pd_resp
mlx5/mlx5_ib.o       struct mlx5_ib_alloc_ucontext_req_v2
mlx5/mlx5_ib.o       struct mlx5_ib_alloc_ucontext_resp
mlx5/mlx5_ib.o       struct mlx5_ib_create_cq
mlx5/mlx5_ib.o       struct mlx5_ib_create_qp
mlx5/mlx5_ib.o       struct mlx5_ib_create_qp_resp
mlx5/mlx5_ib.o       struct mlx5_ib_create_srq
mlx5/mlx5_ib.o       struct mlx5_ib_resize_cq

mthca/ib_mthca.o     struct mthca_alloc_ucontext_resp
mthca/ib_mthca.o     struct mthca_create_cq
mthca/ib_mthca.o     struct mthca_create_qp
mthca/ib_mthca.o     struct mthca_create_srq
mthca/ib_mthca.o     struct mthca_reg_mr
mthca/ib_mthca.o     struct mthca_resize_cq

nes/iw_nes.o         struct nes_alloc_pd_resp
nes/iw_nes.o         struct nes_alloc_ucontext_req
nes/iw_nes.o         struct nes_alloc_ucontext_resp
nes/iw_nes.o         struct nes_create_cq_req
nes/iw_nes.o         struct nes_create_cq_resp
nes/iw_nes.o         struct nes_create_qp_req
nes/iw_nes.o         struct nes_create_qp_resp
nes/iw_nes.o         struct nes_mem_reg_req

ocrdma/ocrdma.o      struct ocrdma_alloc_pd_uresp
ocrdma/ocrdma.o      struct ocrdma_alloc_ucontext_resp
ocrdma/ocrdma.o      struct ocrdma_create_cq_ureq
ocrdma/ocrdma.o      struct ocrdma_create_cq_uresp
ocrdma/ocrdma.o      struct ocrdma_create_qp_ureq
ocrdma/ocrdma.o      struct ocrdma_create_qp_uresp
ocrdma/ocrdma.o      struct ocrdma_create_srq_uresp

usnic/usnic_verbs.o  struct usnic_ib_create_qp_cmd
usnic/usnic_verbs.o  struct usnic_ib_create_qp_resp
usnic/usnic_verbs.o  struct usnic_transport_spec

It seems that amso1100/iw_c2.o, ipath/ib_ipath.o and qib/ib_qib.o don't make use of structure to exchange data with userspace: they use single values, either u32 or u64.

So using pahole I've found issues in mlx5 and cxgb4 only.

BTW, as I don't have the hardware / I don't have access to the hardware, the patches are not tested against real world. (I only have HCAs handled by mlx4 and qib drivers).

Regards.

Yann Droneaud
OPTEYA


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

i386 ABI disagree with most other ABIs regarding alignment of data type larger than 4 bytes: on most ABIs a padding must be added at end of the structures, while it is not required on i386.

So for most ABI struct mlx5_ib_create_srq get implicitly padded to be aligned on a 8 bytes multiple, while for i386, such padding is not added.

Tool pahole could be used to find such implicit padding:

  $ pahole --anon_include \
           --nested_anon_include \
           --recursive \
           --class_name mlx5_ib_create_srq \
           drivers/infiniband/hw/mlx5/mlx5_ib.o

Then, structure layout can be compared between i386 and x86_64:

  +++ obj-i386/drivers/infiniband/hw/mlx5/mlx5_ib.o.pahole.txt    2014-03-28 11:43:07.386413682 +0100
  --- obj-x86_64/drivers/infiniband/hw/mlx5/mlx5_ib.o.pahole.txt  2014-03-27 13:06:17.788472721 +0100
  @@ -69,7 +68,6 @@ struct mlx5_ib_create_srq {
          __u64                      db_addr;              /*     8     8 */
          __u32                      flags;                /*    16     4 */

  -       /* size: 20, cachelines: 1, members: 3 */
  -       /* last cacheline: 20 bytes */
  +       /* size: 24, cachelines: 1, members: 3 */
  +       /* padding: 4 */
  +       /* last cacheline: 24 bytes */

};

ABI disagreement will make an x86_64 kernel try to read past the buffer provided by an i386 binary.

When boundary check will be implemented, the x86_64 kernel will refuse to read past the i386 userspace provided buffer and the uverb will fail.

Anyway, if the structure lay in memory on a page boundary and next page is not mapped, ib_copy_from_udata() will fail and the uverb will fail.

This patch makes create_srq_user() takes care of the input data size to handle the case where no padding was provided.

This way, x86_64 kernel will be able to handle struct mlx5_ib_create_srq as sent by unpatched and patched i386 libmlx5.

---
 drivers/infiniband/hw/mlx5/srq.c  | 18 +++++++++++++++---
 drivers/infiniband/hw/mlx5/user.h |  1 +
 2 files changed, 16 insertions(+), 3 deletions(-)

index 210b3eaf188a..38ca52181eb6 100644
@@ -35,6 +35,7 @@
 #include <linux/mlx5/srq.h>
 #include <linux/slab.h>
 #include <rdma/ib_umem.h>
+#include <rdma/ib_user_verbs.h>
 
 #include "mlx5_ib.h"
 #include "user.h"

@@ -84,10 +85,21 @@ static int create_srq_user(struct ib_pd *pd, struct mlx5_ib_srq *srq,
 	int ncont;
 	u32 offset;
 
-	if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd))) {
-		mlx5_ib_dbg(dev, "failed copy udata\n");
-		return -EFAULT;
+	if (udata->inlen - sizeof(struct ib_uverbs_cmd_hdr) < sizeof(ucmd)) {
+		if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd) - sizeof(ucmd.reserved))) {
+			mlx5_ib_dbg(dev, "failed copy udata\n");
+			return -EFAULT;
+		}
+	} else {
+		if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd))) {
+			mlx5_ib_dbg(dev, "failed copy udata\n");
+			return -EFAULT;
+		}
+
+		if (ucmd.reserved != 0)
+			return -EINVAL;
 	}
+
 	srq->wq_sig = !!(ucmd.flags & MLX5_SRQ_FLAG_SIGNATURE);
 
 	srq->umem = ib_umem_get(pd->uobject->context, ucmd.buf_addr, buf_size,

index d44ecd2c2faf..d0ba264ac1ed 100644
@@ -110,6 +110,7 @@ struct mlx5_ib_create_srq {
 	__u64	buf_addr;
 	__u64	db_addr;
 	__u32	flags;
+	__u32	reserved; /* explicit padding (optional on i386) */
 };
 
 struct mlx5_ib_create_srq_resp {

1.9.0

Hi,

Thanks for the reminder.

The patches were waiting for some times in my patch queue, I failed to provide them earlier. BTW, I've taken time during the week end to complete them. They should be available on [email protected]. See http://marc.info/[email protected]

Regards.

Yann Droneaud
OPTEYA