NETLINK_UESTABLISHED notifier event

This patch provides:

* new event for unicast connections NETLINK_UESTABLISHED.

Quoted 3 lines · Dmitry Yusupov

This patch provides:

* new event for unicast connections NETLINK_UESTABLISHED.

Which kind of application takes benefit of this event ?

btw your diff is reversed ;-)

Jean-Mickael

Quoted 3 lines · Dmitry Yusupov

This patch provides:

* new event for unicast connections NETLINK_UESTABLISHED.

Which kind of application takes benefit of this event ?

btw your diff is reversed ;-)

Jean-Mickael

* new event for unicast connections NETLINK_UESTABLISHED.

* netlink alloc_skb() now uses sk_allocation instead of hard-coded GFP_KERNEL

* since netlink event described via proto and pid, netlink_getsockbypid() is exported, so netlink user can identify socket.

Quoted 2 lines · Dmitry Yusupov

* new event for unicast connections NETLINK_UESTABLISHED.

Huh? In the patch you're actually sending the notification when the socket stops listening to multicast traffic.

Please document why we need this in greater detail too.

Quoted 2 lines

* netlink alloc_skb() now uses sk_allocation instead of hard-coded GFP_KERNEL

Why? We never set it to anything else for netlink.

main reason for this is to have clear way to notify netlink user that new socket created and bound.

one reason is for consistency with sock interface. sk_allocation is equal to GFP_KERNEL by default, so nothing changed. but. in some cases application might require non-blocking kmalloc behavior. one real life example is networking block device used for swap partition. this way any GFP_KERENL allocation on recovery path might lead to deadlock condition.

Quoted 3 lines · Dmitry Yusupov

main reason for this is to have clear way to notify netlink user that new socket created and bound.

Why do you want to exclude sockets that listen to multicast messages? They can still do unicast transactions.

Quoted 5 lines

one reason is for consistency with sock interface. sk_allocation is equal to GFP_KERNEL by default, so nothing changed. but. in some cases application might require non-blocking kmalloc behavior. one real life example is networking block device used for swap partition. this way any GFP_KERENL allocation on recovery path might lead to deadlock condition.

Setting of sk_allocation is controlled by the protocol itself. In this case this is af_netlink.c. As you can see, we never set this to anything other than GFP_KERNEL.

Using sk_allocation will only confuse those who are not familiar with netlink into thinking that this can be atomic.

Cheers,

Quoted 2 lines · Dmitry Yusupov

* netlink alloc_skb() now uses sk_allocation instead of hard-coded GFP_KERNEL

Why? We never set it to anything else for netlink.

It is due to where it is being used. open-iscsi uses netlink sockets for communication in a block (scsi specificically) driver that has pushed much of its code to usersapce. Forcing open-iscsi to use GFP_KERNEL causes a couple of problems. The worst would be where a GFP_KERNEL allocation causes a write, and that write is to an iscsi disk that open-iscsi is managing. The write could then hit the same code path and cause another GFP_KERNEL allocation and we could loop like that until the system locks up.

Quoted 10 lines · Mike Christie

It is due to where it is being used. open-iscsi uses netlink sockets for communication in a block (scsi specificically) driver that has pushed much of its code to usersapce. Forcing open-iscsi to use GFP_KERNEL causes a couple of problems. The worst would be where a GFP_KERNEL allocation causes a write, and that write is to an iscsi disk that open-iscsi is managing. The write could then hit the same code path and cause another GFP_KERNEL allocation and we could loop like that until the system locks up.

In that case it's not enough to just use sk_allocation here. You'll need a way to actually set it to GFP_ATOMIC.

Quoted 3 lines

In that case it's not enough to just use sk_allocation here. You'll need a way to actually set it to GFP_ATOMIC.

That's why in my patch I provided NETLINK_UESTABLISHED event. It is a right way and time to set sk->sk_allocation to GFP_ATOMIC for newly established netlink connection. imho.

Quoted 7 lines · Dmitry Yusupov

Quoted 3 lines

In that case it's not enough to just use sk_allocation here. You'll need a way to actually set it to GFP_ATOMIC.

That's why in my patch I provided NETLINK_UESTABLISHED event. It is a right way and time to set sk->sk_allocation to GFP_ATOMIC for newly established netlink connection. imho.

Right. In that case I think you're going about this the wrong way. It's unmanageable to have some external kernel module fiddle with netlink's sk_allocation upon the receipt of a notification.

If you really want to do this, do it as a setsockopt. It's the owner of the socket that wants the GFP_ATOMIC, not anybody else.

However, I'd like to know more about how open-iscsi uses this. In particular, what are these messages and what do you do when the GFP_ATOMIC alloc_skb fails?

(see Open-iSCSI interface).

Quoted 1 line · Dmitry Yusupov

(see Open-iSCSI interface).

url?

cheers,
jamal

Messages from kernel are asynchronous and there are no alloc_skb on "up" calls. It is "mempooled out" on interface level. (see Open-iSCSI interface). Messages to kernel requires copy_from_user to newly allocated skb, here is where we need sk_allocation bit set. Those messages are synchronous from daemon perspective. If "down" call fails, we will re-try later or take some other management action. We assuming that later OOM-killer will free some memory for us and atomic allocation will succeed eventually.

Quoted 9 lines · Dmitry Yusupov

Messages from kernel are asynchronous and there are no alloc_skb on "up" calls. It is "mempooled out" on interface level. (see Open-iSCSI interface). Messages to kernel requires copy_from_user to newly allocated skb, here is where we need sk_allocation bit set. Those messages are synchronous from daemon perspective. If "down" call fails, we will re-try later or take some other management action. We assuming that later OOM-killer will free some memory for us and atomic allocation will succeed eventually.

I presume you only need to send one message at a time of a fixed size. Would it better to always have an skb allocated for the socket so that we don't need to allocate at sendmsg time?

This actually even better since we will guarantee "down" call delivery. My only concern is that it is not very generic. Do you know clean way to implement it?

Quoted 4 lines · Dmitry Yusupov

This actually even better since we will guarantee "down" call delivery. My only concern is that it is not very generic. Do you know clean way to implement it?

How generic do you want this? Do you need this for socket types other than netlink?

For a one-packet version, we can pre-allocate an skb/page in response to a setsockopt and store it in sk_send_head/sk_sndmsg_page.

This can then be used at sendmsg time. Obviously subsequent messages will have to use alloc_skb until that skb is released.

Alternatively, we can let the socket allocate skb's from an emergency pool similar to what was discussed in the thread
"Summary of 2005 Kernel Summit Proposed Topics".

Again this could enabled on a setsockopt.

Cheers,

This is ideally how it should be for sendmsg paths. socket applications like iscsi, nbd, etc will use it for TCP/IP type of socket. iscsi could re-use the same generic "emergency pool" code for netlink.

Quoted 4 lines · Dmitry Yusupov

This is ideally how it should be for sendmsg paths. socket applications like iscsi, nbd, etc will use it for TCP/IP type of socket. iscsi could re-use the same generic "emergency pool" code for netlink.

In that case we'll wait for the resolution of the discussion on TCP itself, OK?

"emergency pool" is a big and generic feature, needs design and resolution. it could definitely wait. Do you have something against one liner patch for using sk->sk_allocation instead of hard-coded vm priority?