Skip to contentSkip to conversation

queue buffer size increasing howto?

DISCUSSION Banszki Gabor 2 participants 6 messages Aug 22, 2004 – Sep 8, 2004 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

Hi everybody!

Greetings for you! I am a new member of this list.

Please somebody help me!

I am developing an application for counting and processing packets in a linux box with using the ip_queue module.

It is working properly, but sometimes I have a problem, that I get often this message:

Failed to receive netlink message, No buffer space available

Maybee the cause of this, that the box has a lot of short hardly tasks sometimes.
I think the solution to prevent the packet loss If I would increase the queue length. I did it with this command:

echo 2048 > /proc/sys/net/ipv4/ip_queue_maxlen

#cat /proc/net/ip_queue
Peer PID : 23985
Copy mode : 2
Copy range : 2048
Queue length : 184
Queue max. length : 2048

The enviroment:
2.4.26 kernel and iptables: 1.2.9 with promiscuous patch 1.2.3

With this command: watch -n 0 "cat /proc/net/ip_queue" during my application running the biggest number of "Queue length" is 255.

I don't understand it, because the "Queue max. length" is 2048.

How can I increase the queue length?

What is the really method of the queue increasing?

Thanx:

Gabor

#2 lore raw
In reply to a message not in the archive
Quoted 7 lines · Banszki Gabor

Hi everybody!

Greetings for you! I am a new member of this list.

Please somebody help me!

Quoted 7 lines

I am developing an application for counting and processing packets in a linux box with using the ip_queue module.

It is working properly, but sometimes I have a problem, that I get often this message:

Failed to receive netlink message, No buffer space available

This message indicates the the buffer for the netlink socket was full. This is different from the queue that ip_queue has internally for packets to be sent to userspace. You will need to open the netlink socket with a larger buffer, which can be accomplished by using setsockopt(2):

int fd, rv, size = LARGE_SOCKET_BUFFER;

fd = socket( PF_NETLINK, SOCK_RAW, NETLINK_FIREWALL); rv = setsockopt( fd, SOL_SOCKET, SO_RCVBUF, &size, sizeof( size));
...

Quoted 4 lines

With this command: watch -n 0 "cat /proc/net/ip_queue" during my application running the biggest number of "Queue length" is 255.

I don't understand it, because the "Queue max. length" is 2048.

See above. You should be able to increase the size of the socket receive buffer in your userspace process without changing the length of ip_queue's internal packet queue, but test that out to make sure. You will probably need to run the process with root privileges to change your socket receive buffer size, however.

--
[ Tobias DiPasquale ]
0x636f6465736c696e67657240676d61696c2e636f6d

#4 lore raw
In reply to a message not in the archive

Hi Everybody!

Thank for answer of Tobias!!

I could get the buffer size by the getsockopt (107520).

After this I was able to increase the size of buffer to 215040 by your suggestion:

   int size, size_len;
   size_len = sizeof(size);
   rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
   printf ("Oldsize of buffer: %i \n",size); { I've got 107520)

   size = 1256000;
   rv = setsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, size_len); 

   rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
   printf ("New size of buffer: %i \n",size); ( I've got 215040 )

After this I can see the same situation with this command....

watch -n 0 "cat /proc/net/ip_queue"

...like earlier...
The maximum number is around 252 - 258, and I still get the netlink error message "No buffer space....."

I think it didn't help... Why?
Maybee I should increase the SO_RCVBUF by more? But how?

Thnx..

G.

Quoted 19 lines · Tobias DiPasquale

On Sun, 22 Aug 2004 15:40:16 +0200, Banszki Gabor
setsockopt(2):

int fd, rv, size = LARGE_SOCKET_BUFFER;

fd = socket( PF_NETLINK, SOCK_RAW, NETLINK_FIREWALL); rv = setsockopt( fd, SOL_SOCKET, SO_RCVBUF, &size, sizeof( size));
...

Quoted 4 lines

With this command: watch -n 0 "cat /proc/net/ip_queue" during my application running the biggest number of "Queue length" is 255.

I don't understand it, because the "Queue max. length" is 2048.

See above. You should be able to increase the size of the socket receive buffer in your userspace process without changing the length of ip_queue's internal packet queue, but test that out to make sure. You will probably need to run the process with root privileges to change your socket receive buffer size, however.

#5 lore raw

Re: queue buffer size increasing howto? - solved

Hi everybody!

I solved the problem by using pthread.h

The main thread in my process is reading from the queue buffer and open new threads for the long time tasks.

In this case I don't need to increase the buffer size of the netlink socket.

G.

Quoted 56 lines · Banszki Gabor

Hi Everybody!

Thank for answer of Tobias!!

I could get the buffer size by the getsockopt (107520).

After this I was able to increase the size of buffer to 215040 by your suggestion:

   int size, size_len;
   size_len = sizeof(size);
   rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
   printf ("Oldsize of buffer: %i \n",size); { I've got 107520)

   size = 1256000;
   rv = setsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, size_len); 

   rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
   printf ("New size of buffer: %i \n",size); ( I've got 215040 )

After this I can see the same situation with this command....

watch -n 0 "cat /proc/net/ip_queue"

...like earlier...
The maximum number is around 252 - 258, and I still get the netlink error message "No buffer space....."

I think it didn't help... Why?
Maybee I should increase the SO_RCVBUF by more? But how?

Thnx..
other tasks
G.

Quoted 19 lines · Tobias DiPasquale

On Sun, 22 Aug 2004 15:40:16 +0200, Banszki Gabor
setsockopt(2):

int fd, rv, size = LARGE_SOCKET_BUFFER;

fd = socket( PF_NETLINK, SOCK_RAW, NETLINK_FIREWALL); rv = setsockopt( fd, SOL_SOCKET, SO_RCVBUF, &size, sizeof( size));
...

Quoted 4 lines

With this command: watch -n 0 "cat /proc/net/ip_queue" during my application running the biggest number of "Queue length" is 255.

I don't understand it, because the "Queue max. length" is 2048.

See above. You should be able to increase the size of the socket receive buffer in your userspace process without changing the length of ip_queue's internal packet queue, but test that out to make sure. You will probably need to run the process with root privileges to change your socket receive buffer size, however.

#6 lore raw

Re: queue buffer size increasing howto? - solved

Quoted 9 lines · Banszki Gabor

Hi everybody!

I solved the problem by using pthread.h

The main thread in my process is reading from the queue buffer and open new threads for the long time tasks.

In this case I don't need to increase the buffer size of the netlink socket.

You will still lose some data under high enough load. This is exactly what I did, as well, and I still needed to pump up the size of the socket recv buffer.

Signature · 2 lines
[ Tobias DiPasquale ]
0x636f6465736c696e67657240676d61696c2e636f6d
#3 lore raw
In reply to Tobias DiPasquale
Quoted 7 lines · Tobias DiPasquale
Quoted 5 lines · Banszki Gabor

Hi everybody!

I solved the problem by using pthread.h

The main thread in my process is reading from the queue buffer and

open

Quoted 8 lines
Quoted 4 lines

new threads for the long time tasks.

In this case I don't need to increase the buffer size of the netlink socket.

You will still lose some data under high enough load. This is exactly what I did, as well, and I still needed to pump up the size of the socket recv buffer.

Ok....... Bot how can I increase the buffer size by more than 215040?

Quoted 32 lines

Hi Everybody!

Thank for answer of Tobias!!

I could get the buffer size by the getsockopt (107520).

After this I was able to increase the size of buffer to 215040 by your suggestion:

  int size, size_len;
  size_len = sizeof(size);
  rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
  printf ("Oldsize of buffer: %i \n",size); { I've got 107520)

  size = 1256000;
  rv = setsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, size_len); 

  rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
  printf ("New size of buffer: %i \n",size); ( I've got 215040 )

After this I can see the same situation with this command....

watch -n 0 "cat /proc/net/ip_queue"

...like earlier...
The maximum number is around 252 - 258, and I still get the netlink error message "No buffer space....."

I think it didn't help... Why?
Maybee I should increase the SO_RCVBUF by more? But how?

Thnx..