Skip to contentSkip to conversation

netfilter: account ebt_table_info to kmemcg

PATCH In review Shakeel Butt 5 participants 9 messages Jan 3, 2019 – Jan 10, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 netfilter: account ebt_table_info to kmemcg

The [ip,ip6,arp]_tables use x_tables_info internally and the underlying memory is already accounted to kmemcg. Do the same for ebtables. The syzbot, by using setsockopt(EBT_SO_SET_ENTRIES), was able to OOM the whole system from a restricted memcg, a potential DoS.

By accounting the ebt_table_info, the memory used for ebt_table_info can be contained within the memcg of the allocating process. However the lifetime of ebt_table_info is independent of the allocating process and is tied to the network namespace. So, the oom-killer will not be able to relieve the memory pressure due to ebt_table_info memory. The memory for ebt_table_info is allocated through vmalloc. Currently vmalloc does not handle the oom-killed allocating process correctly and one large allocation can bypass memcg limit enforcement. So, with this patch, at least the small allocations will be contained. For large allocations, we need to fix vmalloc.

Reported-by [email protected]
Cc Florian Westphal <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Kirill Tkhai <[email protected]>
Cc Pablo Neira Ayuso <[email protected]>
Cc Jozsef Kadlecsik <[email protected]>
Cc Roopa Prabhu <[email protected]>
Cc Nikolay Aleksandrov <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Linux MM <[email protected]>

---
Changelog since v1:
- More descriptive commit message.

 net/bridge/netfilter/ebtables.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
net/bridge/netfilter/ebtables.c +4 -2
index 491828713e0b..5e55cef0cec3 100644
@@ -1137,14 +1137,16 @@ static int do_replace(struct net *net, const void __user *user,
 	tmp.name[sizeof(tmp.name) - 1] = 0;
 
 	countersize = COUNTER_OFFSET(tmp.nentries) * nr_cpu_ids;
-	newinfo = vmalloc(sizeof(*newinfo) + countersize);
+	newinfo = __vmalloc(sizeof(*newinfo) + countersize, GFP_KERNEL_ACCOUNT,
+			    PAGE_KERNEL);
 	if (!newinfo)
 		return -ENOMEM;
 
 	if (countersize)
 		memset(newinfo->counters, 0, countersize);
 
-	newinfo->entries = vmalloc(tmp.entries_size);
+	newinfo->entries = __vmalloc(tmp.entries_size, GFP_KERNEL_ACCOUNT,
+				     PAGE_KERNEL);
 	if (!newinfo->entries) {
 		ret = -ENOMEM;
 		goto free_newinfo;
Signature · 1 line
2.20.1.415.g653613c723-goog
#1 lore raw
Quoted 17 lines · Shakeel Butt
	countersize = COUNTER_OFFSET(tmp.nentries) * nr_cpu_ids;
net/bridge/netfilter/ebtables.c +2 -1
-	newinfo = vmalloc(sizeof(*newinfo) + countersize);
+	newinfo = __vmalloc(sizeof(*newinfo) + countersize, GFP_KERNEL_ACCOUNT,
+			    PAGE_KERNEL);
	if (!newinfo)
		return -ENOMEM;

	if (countersize)
		memset(newinfo->counters, 0, countersize);
net/bridge/netfilter/ebtables.c +2 -1
-	newinfo->entries = vmalloc(tmp.entries_size);
+	newinfo->entries = __vmalloc(tmp.entries_size, GFP_KERNEL_ACCOUNT,
+				     PAGE_KERNEL);
Quoted 4 lines
	if (!newinfo->entries) {
		ret = -ENOMEM;
		goto free_newinfo;

Just out of curiosity, what are the actual sizes of these areas in typical use given __vmalloc() will be allocating by the page?

#2 lore raw
Quoted 23 lines · William Kucharski
Quoted 12 lines · Shakeel Butt
      countersize = COUNTER_OFFSET(tmp.nentries) * nr_cpu_ids;
net/bridge/netfilter/ebtables.c +2 -1
-     newinfo = vmalloc(sizeof(*newinfo) + countersize);
+     newinfo = __vmalloc(sizeof(*newinfo) + countersize, GFP_KERNEL_ACCOUNT,
+                         PAGE_KERNEL);
      if (!newinfo)
              return -ENOMEM;
 

net/bridge/netfilter/ebtables.c static int do_replace(struct net *net, const void __user *user,

Quoted 10 lines
net/bridge/netfilter/ebtables.c +2 -2
      if (countersize)
              memset(newinfo->counters, 0, countersize);
 
-     newinfo->entries = vmalloc(tmp.entries_size);
+     newinfo->entries = __vmalloc(tmp.entries_size, GFP_KERNEL_ACCOUNT,
+                                  PAGE_KERNEL);
      if (!newinfo->entries) {
              ret = -ENOMEM;
              goto free_newinfo;
--
Quoted 3 lines

Just out of curiosity, what are the actual sizes of these areas in typical use given __vmalloc() will be allocating by the page?

We don't really use this in production, so, I don't have a good idea of the size in the typical case. The size depends on the workload. The motivation behind this patch was the system OOM triggered by a syzbot running in a restricted memcg.

Shakeel

Reviewed-by on v2 1/1
#3 lore raw
In reply to Shakeel Butt
Quoted 31 lines · Shakeel Butt

The [ip,ip6,arp]_tables use x_tables_info internally and the underlying memory is already accounted to kmemcg. Do the same for ebtables. The syzbot, by using setsockopt(EBT_SO_SET_ENTRIES), was able to OOM the whole system from a restricted memcg, a potential DoS.

By accounting the ebt_table_info, the memory used for ebt_table_info can be contained within the memcg of the allocating process. However the lifetime of ebt_table_info is independent of the allocating process and is tied to the network namespace. So, the oom-killer will not be able to relieve the memory pressure due to ebt_table_info memory. The memory for ebt_table_info is allocated through vmalloc. Currently vmalloc does not handle the oom-killed allocating process correctly and one large allocation can bypass memcg limit enforcement. So, with this patch, at least the small allocations will be contained. For large allocations, we need to fix vmalloc.

Reported-by [email protected]
Cc Florian Westphal <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Kirill Tkhai <[email protected]>
Cc Pablo Neira Ayuso <[email protected]>
Cc Jozsef Kadlecsik <[email protected]>
Cc Roopa Prabhu <[email protected]>
Cc Nikolay Aleksandrov <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Linux MM <[email protected]>
Quoted 3 lines

---
Changelog since v1:
- More descriptive commit message.

Quoted 28 lines
 net/bridge/netfilter/ebtables.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
net/bridge/netfilter/ebtables.c +4 -2
index 491828713e0b..5e55cef0cec3 100644
@@ -1137,14 +1137,16 @@ static int do_replace(struct net *net, const void __user *user,
 	tmp.name[sizeof(tmp.name) - 1] = 0;
 
 	countersize = COUNTER_OFFSET(tmp.nentries) * nr_cpu_ids;
-	newinfo = vmalloc(sizeof(*newinfo) + countersize);
+	newinfo = __vmalloc(sizeof(*newinfo) + countersize, GFP_KERNEL_ACCOUNT,
+			    PAGE_KERNEL);
 	if (!newinfo)
 		return -ENOMEM;
 
 	if (countersize)
 		memset(newinfo->counters, 0, countersize);
 
-	newinfo->entries = vmalloc(tmp.entries_size);
+	newinfo->entries = __vmalloc(tmp.entries_size, GFP_KERNEL_ACCOUNT,
+				     PAGE_KERNEL);
 	if (!newinfo->entries) {
 		ret = -ENOMEM;
 		goto free_newinfo;
#4 lore raw
Quoted 49 lines · Kirill Tkhai
Quoted 31 lines · Shakeel Butt

The [ip,ip6,arp]_tables use x_tables_info internally and the underlying memory is already accounted to kmemcg. Do the same for ebtables. The syzbot, by using setsockopt(EBT_SO_SET_ENTRIES), was able to OOM the whole system from a restricted memcg, a potential DoS.

By accounting the ebt_table_info, the memory used for ebt_table_info can be contained within the memcg of the allocating process. However the lifetime of ebt_table_info is independent of the allocating process and is tied to the network namespace. So, the oom-killer will not be able to relieve the memory pressure due to ebt_table_info memory. The memory for ebt_table_info is allocated through vmalloc. Currently vmalloc does not handle the oom-killed allocating process correctly and one large allocation can bypass memcg limit enforcement. So, with this patch, at least the small allocations will be contained. For large allocations, we need to fix vmalloc.

Reported-by [email protected]
Cc Florian Westphal <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Kirill Tkhai <[email protected]>
Cc Pablo Neira Ayuso <[email protected]>
Cc Jozsef Kadlecsik <[email protected]>
Cc Roopa Prabhu <[email protected]>
Cc Nikolay Aleksandrov <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Linux MM <[email protected]>
Quoted 3 lines

---
Changelog since v1:
- More descriptive commit message.

Quoted 11 lines
 net/bridge/netfilter/ebtables.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
net/bridge/netfilter/ebtables.c +0 -0
index 491828713e0b..5e55cef0cec3 100644
@@ -1137,14 +1137,16 @@ static int do_replace(struct net *net, const void __user *user,
 	tmp.name[sizeof(tmp.name) - 1] = 0;
 
Quoted 4 lines

net/bridge/netfilter/ebtables.c static int do_replace(struct net *net, const void __user *user, · line 1141

Quoted 4 lines
net/bridge/netfilter/ebtables.c +2 -1
 	countersize = COUNTER_OFFSET(tmp.nentries) * nr_cpu_ids;
-	newinfo = vmalloc(sizeof(*newinfo) + countersize);
+	newinfo = __vmalloc(sizeof(*newinfo) + countersize, GFP_KERNEL_ACCOUNT,
+			    PAGE_KERNEL);

Do we need GFP_HIGHMEM here?

Quoted 14 lines
Quoted 13 lines
net/bridge/netfilter/ebtables.c +2 -1
 	if (!newinfo)
 		return -ENOMEM;
 
 	if (countersize)
 		memset(newinfo->counters, 0, countersize);
 
-	newinfo->entries = vmalloc(tmp.entries_size);
+	newinfo->entries = __vmalloc(tmp.entries_size, GFP_KERNEL_ACCOUNT,
+				     PAGE_KERNEL);
 	if (!newinfo->entries) {
 		ret = -ENOMEM;
 		goto free_newinfo;
#5 lore raw

On Thu 10-01-19 12:22:09, Kirill Tkhai wrote:
[...]

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
net/bridge/netfilter/ebtables.c +0 -0
index 491828713e0b..5e55cef0cec3 100644
@@ -1137,14 +1137,16 @@ static int do_replace(struct net *net, const void __user *user,
 	tmp.name[sizeof(tmp.name) - 1] = 0;
 
Quoted 6 lines
Quoted 4 lines

net/bridge/netfilter/ebtables.c static int do_replace(struct net *net, const void __user *user, · line 1141

Quoted 4 lines
net/bridge/netfilter/ebtables.c +2 -1
 	countersize = COUNTER_OFFSET(tmp.nentries) * nr_cpu_ids;
-	newinfo = vmalloc(sizeof(*newinfo) + countersize);
+	newinfo = __vmalloc(sizeof(*newinfo) + countersize, GFP_KERNEL_ACCOUNT,
+			    PAGE_KERNEL);

Do we need GFP_HIGHMEM here?

No. vmalloc adds __GPF_HIGHMEM implicitly (see __vmalloc_area_node).

Signature · 2 lines
Michal Hocko
SUSE Labs
#6 lore raw
Quoted 9 lines · Michal Hocko

On Thu 10-01-19 12:22:09, Kirill Tkhai wrote:
[...]

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
net/bridge/netfilter/ebtables.c +0 -0
index 491828713e0b..5e55cef0cec3 100644
@@ -1137,14 +1137,16 @@ static int do_replace(struct net *net, const void __user *user,
 	tmp.name[sizeof(tmp.name) - 1] = 0;
 
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

net/bridge/netfilter/ebtables.c static int do_replace(struct net *net, const void __user *user, · line 1141

Quoted 4 lines
net/bridge/netfilter/ebtables.c +2 -1
 	countersize = COUNTER_OFFSET(tmp.nentries) * nr_cpu_ids;
-	newinfo = vmalloc(sizeof(*newinfo) + countersize);
+	newinfo = __vmalloc(sizeof(*newinfo) + countersize, GFP_KERNEL_ACCOUNT,
+			    PAGE_KERNEL);

Do we need GFP_HIGHMEM here?

No. vmalloc adds __GPF_HIGHMEM implicitly (see __vmalloc_area_node).

Then OK, thanks for the explanation.

Kirill

#7 lore raw
In reply to Shakeel Butt
Quoted 5 lines · Shakeel Butt

The [ip,ip6,arp]_tables use x_tables_info internally and the underlying memory is already accounted to kmemcg. Do the same for ebtables. The syzbot, by using setsockopt(EBT_SO_SET_ENTRIES), was able to OOM the whole system from a restricted memcg, a potential DoS.

Quoted 10 lines

By accounting the ebt_table_info, the memory used for ebt_table_info can be contained within the memcg of the allocating process. However the lifetime of ebt_table_info is independent of the allocating process and is tied to the network namespace. So, the oom-killer will not be able to relieve the memory pressure due to ebt_table_info memory. The memory for ebt_table_info is allocated through vmalloc. Currently vmalloc does not handle the oom-killed allocating process correctly and one large allocation can bypass memcg limit enforcement. So, with this patch, at least the small allocations will be contained. For large allocations, we need to fix vmalloc.

Fine with this -mm?

If no objections, I'll apply this to the netfilter tree. Thanks.

Quoted 48 lines
Reported-by [email protected]
Cc Florian Westphal <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Kirill Tkhai <[email protected]>
Cc Pablo Neira Ayuso <[email protected]>
Cc Jozsef Kadlecsik <[email protected]>
Cc Roopa Prabhu <[email protected]>
Cc Nikolay Aleksandrov <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Linux MM <[email protected]>

---
Changelog since v1:
- More descriptive commit message.

 net/bridge/netfilter/ebtables.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
net/bridge/netfilter/ebtables.c +4 -2
index 491828713e0b..5e55cef0cec3 100644
@@ -1137,14 +1137,16 @@ static int do_replace(struct net *net, const void __user *user,
 	tmp.name[sizeof(tmp.name) - 1] = 0;
 
 	countersize = COUNTER_OFFSET(tmp.nentries) * nr_cpu_ids;
-	newinfo = vmalloc(sizeof(*newinfo) + countersize);
+	newinfo = __vmalloc(sizeof(*newinfo) + countersize, GFP_KERNEL_ACCOUNT,
+			    PAGE_KERNEL);
 	if (!newinfo)
 		return -ENOMEM;
 
 	if (countersize)
 		memset(newinfo->counters, 0, countersize);
 
-	newinfo->entries = vmalloc(tmp.entries_size);
+	newinfo->entries = __vmalloc(tmp.entries_size, GFP_KERNEL_ACCOUNT,
+				     PAGE_KERNEL);
 	if (!newinfo->entries) {
 		ret = -ENOMEM;
 		goto free_newinfo;
Signature · 1 line
2.20.1.415.g653613c723-goog
#8 lore raw
In reply to Shakeel Butt
Quoted 5 lines · Shakeel Butt

The [ip,ip6,arp]_tables use x_tables_info internally and the underlying memory is already accounted to kmemcg. Do the same for ebtables. The syzbot, by using setsockopt(EBT_SO_SET_ENTRIES), was able to OOM the whole system from a restricted memcg, a potential DoS.

Quoted 10 lines

By accounting the ebt_table_info, the memory used for ebt_table_info can be contained within the memcg of the allocating process. However the lifetime of ebt_table_info is independent of the allocating process and is tied to the network namespace. So, the oom-killer will not be able to relieve the memory pressure due to ebt_table_info memory. The memory for ebt_table_info is allocated through vmalloc. Currently vmalloc does not handle the oom-killed allocating process correctly and one large allocation can bypass memcg limit enforcement. So, with this patch, at least the small allocations will be contained. For large allocations, we need to fix vmalloc.

OK, patch is applied, thanks.