ucma question - backlog limit

Hey Sean,

What is the logic behind limiting the listen backlog to 128?

------

snipit from ucma.c:

enum {
UCMA_MAX_BACKLOG = 128
};

snipit from ucma_listen():

         ctx->backlog = cmd.backlog > 0 && cmd.backlog <

                        cmd.backlog : UCMA_MAX_BACKLOG;

------

Thanks,

Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I think that could be relaxed.
Running OFED (and softiwarp) on larger installations, we already had to set this to some KByte...

Bernard.

[email protected] wrote on 08/09/2010 04:34:31 PM:

Quoted 8 lines

Hey Sean,

What is the logic behind limiting the listen backlog to 128?

------

snipit from ucma.c:

Quoted 9 lines

enum {
UCMA_MAX_BACKLOG = 128
};

snipit from ucma_listen():

          ctx->backlog = cmd.backlog>  0&&  cmd.backlog<

UCMA_MAX_BACKLOG ?

Quoted 5 lines

                         cmd.backlog : UCMA_MAX_BACKLOG;

------

Quoted 8 lines

Thanks,

Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I agree. I think it should default to say 1024. Further, I propose we make this a sysctl variable such that admins can tweak it as needed.

Quoted 3 lines

I agree. I think it should default to say 1024. Further, I propose we make this a sysctl variable such that admins can tweak it as needed.

The default is based on:

#define SOMAXCONN 128

I have no objection to making it adjustable.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

For TCP, a dropped SYN will cause the peer to retransmit the SYN, and the connection will be eventually established. So a small backlog limit is probably ok for TCP. With iWARP, the connect request is TCP payload an on established connection. So the connection request is transmitted and acked at the TCP level by the time the connect request gets dropped in the ucma. What finally happens is the peer times out (after many seconds typically) and kills the connection. Further, a 32 node 256NP OpenMPI job will generate > 128 connect requests on some ranks.

So I think its reasonable to up default max value. I'll post a patch
to change the max and make it a sysctl variable as well.

Ok?

Quoted 1 line

Quoted 1 line

I agree. I think it should default to say 1024. Further, I propose

Quoted 11 lines

Quoted 2 lines

make this a sysctl variable such that admins can tweak it as needed.

The default is based on:

#define SOMAXCONN 128

I have no objection to making it adjustable.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma"

Quoted 3 lines

the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

For TCP, a dropped SYN will cause the peer to retransmit the SYN, and the connection will be eventually established. So a small backlog limit is probably ok for TCP. With iWARP, the connect request is TCP payload an on established connection. So the connection request is transmitted and acked at the TCP level by the time the connect request gets dropped in the ucma. What finally happens is the peer times out (after many seconds typically) and kills the connection. Further, a 32 node 256NP OpenMPI job will generate > 128 connect requests on some ranks.

So I think its reasonable to up default max value. I'll post a patch
to change the max and make it a sysctl variable as well.

Ok?

Steve.