Skip to contentSkip to conversation

mlx4_wq_overflow question

DISCUSSION Eli Cohen 2 participants 2 messages Feb 8, 2012 – Feb 9, 2012 InfiniBand

lore
Share & tools
lore

Eli Cohen

author
#0 lore raw

Hi Roland,

referring to the function below, I can't find a strong reason to check again after acquiring the cq spinlock. Any idea?

static int mlx4_wq_overflow(struct mlx4_ib_wq *wq, int nreq, struct ib_cq *ib_cq)

{
        unsigned cur;
        struct mlx4_ib_cq *cq;

        cur = wq->head - wq->tail;
        if (likely(cur + nreq < wq->max_post))
                return 0;

        cq = to_mcq(ib_cq);
        spin_lock(&cq->lock);
        cur = wq->head - wq->tail;
        spin_unlock(&cq->lock);

        return cur + nreq >= wq->max_post;
}

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#1 lore raw
Quoted 13 lines · Eli Cohen

referring to the function below, I can't find a strong reason to check again after acquiring the cq spinlock. Any idea?

static int mlx4_wq_overflow(struct mlx4_ib_wq *wq, int nreq, struct ib_cq *ib_cq)
{
       unsigned cur;
       struct mlx4_ib_cq *cq;

       cur = wq->head - wq->tail;
       if (likely(cur + nreq < wq->max_post))
               return 0;

Quoted 7 lines

       cq = to_mcq(ib_cq);
       spin_lock(&cq->lock);
       cur = wq->head - wq->tail;
       spin_unlock(&cq->lock);

       return cur + nreq >= wq->max_post;
}

This is the typical optimization of doing a cheaper imprecise check first, and then only if we need to (ie the QP was close to full) take the lock and synchronize with completions being polled to get an exact answer. Without the second check we might have spurious "WQ full" indications.

If I recall correctly, MST first proposed this optimization for mthca years ago. If you search the archives you can probably find his detailed explanation.

- R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html