UDP memory usage accounting: accounting unit and variable
Subsystems (4)
PATCH 2/3 UDP memory usage accounting: accounting unit and variable
This patch introduces global variable for UDP memory accounting. The unit is page.
include/net/sock.h +0 -0
@@ -723,6 +723,13 @@ static inline int sk_stream_wmem_schedul
sk_stream_mem_schedule(sk, size, 0);
}
+#define SK_DATAGRAM_MEM_QUANTUM ((int)PAGE_SIZE) +
+static inline int sk_datagram_pages(int amt) +{
+ return DIV_ROUND_UP(amt, SK_DATAGRAM_MEM_QUANTUM); +}
+
/* Used by processes to "lock" a socket state, so that
* interrupts and bottom half handlers won't change it
* from under us. It essentially blocks any incoming
include/net/udp.h +0 -0
@@ -65,6 +65,8 @@ extern rwlock_t udp_hash_lock;
extern struct proto udp_prot;
+extern atomic_t udp_memory_allocated;
+
struct sk_buff;
/*
net/ipv4/proc.c +0 -0
@@ -66,7 +66,8 @@ static int sockstat_seq_show(struct seq_
fold_prot_inuse(&tcp_prot), atomic_read(&tcp_orphan_count), tcp_death_row.tw_count, atomic_read(&tcp_sockets_allocated), atomic_read(&tcp_memory_allocated));
- seq_printf(seq, "UDP: inuse %d\n", fold_prot_inuse(&udp_prot));
+ seq_printf(seq, "UDP: inuse %d mem %d\n", fold_prot_inuse(&udp_prot),
+ atomic_read(&udp_memory_allocated));seq_printf(seq, "UDPLITE: inuse %d\n", fold_prot_inuse(&udplite_prot)); seq_printf(seq, "RAW: inuse %d\n", fold_prot_inuse(&raw_prot)); seq_printf(seq, "FRAG: inuse %d memory %d\n", ip_frag_nqueues,
net/ipv4/udp.c +0 -0
@@ -113,6 +113,10 @@ DEFINE_SNMP_STAT(struct udp_mib, udp_sta
struct hlist_head udp_hash[UDP_HTABLE_SIZE];DEFINE_RWLOCK(udp_hash_lock);
+atomic_t udp_memory_allocated;
+
+EXPORT_SYMBOL(udp_memory_allocated);
+
static int udp_port_rover;
static inline int __udp_lib_lport_inuse(__u16 num, struct hlist_head udptable[])
Quoted 2 lines · Satoshi OSHIMA
This patch introduces global variable for UDP memory accounting. The unit is page.
The global variable doesn't seem to be very MP scalable, especially if you change it for each packet. This will be a very hot cache line, in the worst case bouncing around a large machine.
Possible alternatives:
- Per CPU variables
- You only change the global on socket creation time (by pre allocating a large amount) or when the system comes under memory pressure.
- Batching of the global updates for multiple packets [that's a variant of the previous one, might be still too costly though]
Also for such variables it's usually good to cache line pad them on SMP to avoid false sharing with something else.
-Andi
Hello,
I apologize for not replying sooner.
Quoted 16 lines · Andi Kleen
Quoted 2 lines · Satoshi OSHIMA
This patch introduces global variable for UDP memory accounting. The unit is page.
The global variable doesn't seem to be very MP scalable, especially if you change it for each packet. This will be a very hot cache line, in the worst case bouncing around a large machine.
Possible alternatives:
- Per CPU variables
- You only change the global on socket creation time (by pre allocating a large amount) or when the system comes under memory pressure.
- Batching of the global updates for multiple packets [that's a variant of the previous one, might be still too costly though]
Quoted 4 lines
Also for such variables it's usually good to cache line pad them on SMP to avoid false sharing with something else.
-Andi
Thank you so much for your suggestions.
The implementation of the patch basically followed implementation of tcp_memory_allocated. However, I should agree that the patch introduces atomic operations too much. Then, I'll try to use the batching to reduce the number of atomic operations.
Best regards,
Hideo Aoki
--
Hitachi Computer Products (America) Inc.
Hi,
Thank you for your comment.
Quoted 8 lines · Andi Kleen
Quoted 2 lines · Satoshi OSHIMA
This patch introduces global variable for UDP memory accounting. The unit is page.
The global variable doesn't seem to be very MP scalable, especially if you change it for each packet. This will be a very hot cache line, in the worst case bouncing around a large machine.
I understand what you pointed out. But I think the accounting method I'm proposing is very similar to TCP accounting and per socket accounting.
How do you think of it?
Quoted 2 lines
Possible alternatives:
- Per CPU variables
I'm afraid that sockets and socket buffers are handled on various CPUs. I mean that socket creation might be done on CPU-A but socket receiving might be done on CPU-B.
And per CPU variables must be counted up when socket cap is checked. I'm afraid that per CPU vaiables are also costly enough.
Quoted 1 line
- You only change the global on socket creation time (by pre
allocating a large
Quoted 6 lines
amount) or when the system comes under memory pressure.
- Batching of the global updates for multiple packets [that's a variant of the previous one, might be still too costly though]Also for such variables it's usually good to cache line pad them on SMP to avoid false sharing with something else.
I believe that memory usage accounting should be done accurately. Currently I couldn't see how can we know the accurate memory accounting only when the system is under memory pressure.
But I revised the patch to avoid some atomic operations.
If I could find the good way to avoid atomic operation more, I will add it.
Satoshi Oshima
Quoted 5 lines · Satoshi OSHIMA
I understand what you pointed out. But I think the accounting method I'm proposing is very similar to TCP accounting and per socket accounting.
How do you think of it?
I think allowing Joe user to stop root from using TCP or UDP isn't much better than having him hog kernel memory.
So let's fix this properly and add per-user limits rather than system-wide ones.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt