Skip to contentSkip to conversation

bpf: set inner_map_meta->spin_lock_off correctly

PATCH Applied Yonghong Song 4 participants 11 messages Feb 27, 2019 – Feb 28, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover bpf: set inner_map_meta->spin_lock_off correctly

The inner_map_meta->spin_lock_off is not set correctly during map creation for BPF_MAP_TYPE_ARRAY_OF_MAPS and BPF_MAP_TYPE_HASH_OF_MAPS. This may lead verifier error due to misinformation.
This patch set fixed the issue with Patch #1 for the kernel change and Patch #2 for enhanced selftest test_maps.

Yonghong Song (2):
bpf: set inner_map_meta->spin_lock_off correctly tools/bpf: selftests: add map lookup to test_map_in_map bpf prog

 kernel/bpf/map_in_map.c                             | 1 +
 tools/testing/selftests/bpf/progs/test_map_in_map.c | 4 ++++
 2 files changed, 5 insertions(+)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/2 bpf: set inner_map_meta->spin_lock_off correctly

Commit d83525ca62cf ("bpf: introduce bpf_spin_lock") introduced bpf_spin_lock and the field spin_lock_off in kernel internal structure bpf_map has the following meaning:

Quoted 1 line

=0 valid offset, <0 error

For every map created, the kernel will ensure spin_lock_off has correct value.

Currently, bpf_map->spin_lock_off is not copied from the inner map to the map_in_map inner_map_meta during a map_in_map type map creation, so inner_map_meta->spin_lock_off = 0.
This will give verifier wrong information that inner_map has bpf_spin_lock and the bpf_spin_lock is defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lock cannot be accessed directly by load/store

This patch fixed the issue by copy inner map's spin_lock_off value to inner_map_meta->spin_lock_off.

Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")
---
 kernel/bpf/map_in_map.c | 1 +
 1 file changed, 1 insertion(+)
kernel/bpf/map_in_map.c +1 -0
index 583346a0ab29..3dff41403583 100644
@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)
 	inner_map_meta->value_size = inner_map->value_size;
 	inner_map_meta->map_flags = inner_map->map_flags;
 	inner_map_meta->max_entries = inner_map->max_entries;
+	inner_map_meta->spin_lock_off = inner_map->spin_lock_off;
 
 	/* Misc members not needed in bpf_map_meta_equal() check. */
 	inner_map_meta->ops = inner_map->ops;
Signature · 1 line
2.17.1
#2 lore raw
Quoted 34 lines · Yonghong Song

Commit d83525ca62cf ("bpf: introduce bpf_spin_lock") introduced bpf_spin_lock and the field spin_lock_off in kernel internal structure bpf_map has the following meaning:

Quoted 1 line

=0 valid offset, <0 error

For every map created, the kernel will ensure spin_lock_off has correct value.

Currently, bpf_map->spin_lock_off is not copied from the inner map to the map_in_map inner_map_meta during a map_in_map type map creation, so inner_map_meta->spin_lock_off = 0.
This will give verifier wrong information that inner_map has bpf_spin_lock and the bpf_spin_lock is defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lock cannot be accessed directly by load/store

This patch fixed the issue by copy inner map's spin_lock_off value to inner_map_meta->spin_lock_off.

Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")
---
 kernel/bpf/map_in_map.c | 1 +
 1 file changed, 1 insertion(+)
kernel/bpf/map_in_map.c +0 -0
index 583346a0ab29..3dff41403583 100644

kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61

Quoted 4 lines
kernel/bpf/map_in_map.c +1 -0
@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)
        inner_map_meta->value_size = inner_map->value_size;
        inner_map_meta->map_flags = inner_map->map_flags;
        inner_map_meta->max_entries = inner_map->max_entries;
+       inner_map_meta->spin_lock_off = inner_map->spin_lock_off;

Looks like spinlock inside inner map is not supported: there is specific check few lines above returning -ENOSUPP for such case. In that case, maybe assign -1 here to make this explicit?

Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?

Quoted 6 lines
kernel/bpf/map_in_map.c +0 -1
        /* Misc members not needed in bpf_map_meta_equal() check. */
        inner_map_meta->ops = inner_map->ops;
--

2.17.1

#3 lore raw
Quoted 30 lines · Andrii Nakryiko
Quoted 29 lines · Yonghong Song

Commit d83525ca62cf ("bpf: introduce bpf_spin_lock") introduced bpf_spin_lock and the field spin_lock_off in kernel internal structure bpf_map has the following meaning:

Quoted 1 line

=0 valid offset, <0 error

For every map created, the kernel will ensure spin_lock_off has correct value.

Currently, bpf_map->spin_lock_off is not copied from the inner map to the map_in_map inner_map_meta during a map_in_map type map creation, so inner_map_meta->spin_lock_off = 0.
This will give verifier wrong information that inner_map has bpf_spin_lock and the bpf_spin_lock is defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lock cannot be accessed directly by load/store

This patch fixed the issue by copy inner map's spin_lock_off value to inner_map_meta->spin_lock_off.

Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")
---
  kernel/bpf/map_in_map.c | 1 +
  1 file changed, 1 insertion(+)
Quoted 13 lines

kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61

Quoted 9 lines
kernel/bpf/map_in_map.c +1 -0
index 583346a0ab29..3dff41403583 100644
@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)
         inner_map_meta->value_size = inner_map->value_size;
         inner_map_meta->map_flags = inner_map->map_flags;
         inner_map_meta->max_entries = inner_map->max_entries;
+       inner_map_meta->spin_lock_off = inner_map->spin_lock_off;

Looks like spinlock inside inner map is not supported: there is specific check few lines above returning -ENOSUPP for such case. In that case, maybe assign -1 here to make this explicit?

-1 (-EPERM) probably not the best choice. The verifier already has knowledge that a particular tracked map is an inner map or not. So keeping the original error code (mostly -EINVAL) is preferred I think.

Quoted 4 lines

Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?

Not exactly sure about the reason. Maybe with this patch, it can get proper support. Not 100% sure.

Quoted 7 lines
Quoted 7 lines
kernel/bpf/map_in_map.c +0 -1
         /* Misc members not needed in bpf_map_meta_equal() check. */
         inner_map_meta->ops = inner_map->ops;
--

2.17.1

#4 lore raw
Quoted 25 lines · Yonghong Song
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Yonghong Song

Commit d83525ca62cf ("bpf: introduce bpf_spin_lock") introduced bpf_spin_lock and the field spin_lock_off in kernel internal structure bpf_map has the following meaning:

Quoted 1 line

=0 valid offset, <0 error

For every map created, the kernel will ensure spin_lock_off has correct value.

Currently, bpf_map->spin_lock_off is not copied from the inner map to the map_in_map inner_map_meta during a map_in_map type map creation, so inner_map_meta->spin_lock_off = 0.
This will give verifier wrong information that inner_map has bpf_spin_lock and the bpf_spin_lock is defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lock cannot be accessed directly by load/store

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

This patch fixed the issue by copy inner map's spin_lock_off value to inner_map_meta->spin_lock_off.

Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")
Quoted 18 lines
Quoted 4 lines
Quoted 4 lines
---
  kernel/bpf/map_in_map.c | 1 +
  1 file changed, 1 insertion(+)
Quoted 13 lines

kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61

Quoted 9 lines
kernel/bpf/map_in_map.c +1 -0
index 583346a0ab29..3dff41403583 100644
@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)
         inner_map_meta->value_size = inner_map->value_size;
         inner_map_meta->map_flags = inner_map->map_flags;
         inner_map_meta->max_entries = inner_map->max_entries;
+       inner_map_meta->spin_lock_off = inner_map->spin_lock_off;

Looks like spinlock inside inner map is not supported: there is specific check few lines above returning -ENOSUPP for such case. In that case, maybe assign -1 here to make this explicit?

Quoted 3 lines

-1 (-EPERM) probably not the best choice. The verifier already has knowledge that a particular tracked map is an inner map or not. So keeping the original error code (mostly -EINVAL) is preferred I think.

Ah, I actually missed the fact that verifier actually checks those values (so it's not just >= 0 or < 0), so yeah, let's just pass through. Btw, the value when there is no spinlock is actually -ENOENT.

Quoted 8 lines
Quoted 5 lines

Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?

Not exactly sure about the reason. Maybe with this patch, it can get proper support. Not 100% sure.

No, it won't, because bpf_map_meta_alloc explicitly tests for it:

        if (map_value_has_spin_lock(inner_map)) {
                fdput(f);
                return ERR_PTR(-ENOTSUPP);
        }

Maybe Alexei can clarify?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/bpf/map_in_map.c +0 -1
         /* Misc members not needed in bpf_map_meta_equal() check. */
         inner_map_meta->ops = inner_map->ops;
--

2.17.1

#5 lore raw
Quoted 52 lines · Andrii Nakryiko
Quoted 25 lines · Yonghong Song
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Yonghong Song

Commit d83525ca62cf ("bpf: introduce bpf_spin_lock") introduced bpf_spin_lock and the field spin_lock_off in kernel internal structure bpf_map has the following meaning:

Quoted 1 line

=0 valid offset, <0 error

For every map created, the kernel will ensure spin_lock_off has correct value.

Currently, bpf_map->spin_lock_off is not copied from the inner map to the map_in_map inner_map_meta during a map_in_map type map creation, so inner_map_meta->spin_lock_off = 0.
This will give verifier wrong information that inner_map has bpf_spin_lock and the bpf_spin_lock is defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lock cannot be accessed directly by load/store

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

This patch fixed the issue by copy inner map's spin_lock_off value to inner_map_meta->spin_lock_off.

Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")
Quoted 18 lines
Quoted 17 lines

kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61

Quoted 13 lines
---
   kernel/bpf/map_in_map.c | 1 +
   1 file changed, 1 insertion(+)
kernel/bpf/map_in_map.c +1 -0
index 583346a0ab29..3dff41403583 100644
@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)
          inner_map_meta->value_size = inner_map->value_size;
          inner_map_meta->map_flags = inner_map->map_flags;
          inner_map_meta->max_entries = inner_map->max_entries;
+       inner_map_meta->spin_lock_off = inner_map->spin_lock_off;

Looks like spinlock inside inner map is not supported: there is specific check few lines above returning -ENOSUPP for such case. In that case, maybe assign -1 here to make this explicit?

Quoted 7 lines
Quoted 3 lines

-1 (-EPERM) probably not the best choice. The verifier already has knowledge that a particular tracked map is an inner map or not. So keeping the original error code (mostly -EINVAL) is preferred I think.

Ah, I actually missed the fact that verifier actually checks those values (so it's not just >= 0 or < 0), so yeah, let's just pass through. Btw, the value when there is no spinlock is actually -ENOENT.

If there is no BTF, it will be -EINVAL. If there is BTF and no spinlock member, mostly -ENOENT.

Quoted 10 lines
Quoted 9 lines
Quoted 6 lines

Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?

Not exactly sure about the reason. Maybe with this patch, it can get proper support. Not 100% sure.

Quoted 6 lines

No, it won't, because bpf_map_meta_alloc explicitly tests for it:

         if (map_value_has_spin_lock(inner_map)) {
                 fdput(f);
                 return ERR_PTR(-ENOTSUPP);
         }

I mean that this can be removed after my patch and it may work :-)

Quoted 12 lines

Maybe Alexei can clarify?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/bpf/map_in_map.c +0 -1
          /* Misc members not needed in bpf_map_meta_equal() check. */
          inner_map_meta->ops = inner_map->ops;
--

2.17.1

#6 lore raw
Quoted 60 lines · Yonghong Song
Quoted 56 lines · Andrii Nakryiko
Quoted 25 lines · Yonghong Song
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Yonghong Song

Commit d83525ca62cf ("bpf: introduce bpf_spin_lock") introduced bpf_spin_lock and the field spin_lock_off in kernel internal structure bpf_map has the following meaning:

Quoted 1 line

=0 valid offset, <0 error

For every map created, the kernel will ensure spin_lock_off has correct value.

Currently, bpf_map->spin_lock_off is not copied from the inner map to the map_in_map inner_map_meta during a map_in_map type map creation, so inner_map_meta->spin_lock_off = 0.
This will give verifier wrong information that inner_map has bpf_spin_lock and the bpf_spin_lock is defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lock cannot be accessed directly by load/store

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

This patch fixed the issue by copy inner map's spin_lock_off value to inner_map_meta->spin_lock_off.

Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")
Quoted 21 lines
Quoted 4 lines
Quoted 4 lines
---
   kernel/bpf/map_in_map.c | 1 +
   1 file changed, 1 insertion(+)
Quoted 13 lines

kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61

Quoted 9 lines
kernel/bpf/map_in_map.c +1 -0
index 583346a0ab29..3dff41403583 100644
@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)
          inner_map_meta->value_size = inner_map->value_size;
          inner_map_meta->map_flags = inner_map->map_flags;
          inner_map_meta->max_entries = inner_map->max_entries;
+       inner_map_meta->spin_lock_off = inner_map->spin_lock_off;

Looks like spinlock inside inner map is not supported: there is specific check few lines above returning -ENOSUPP for such case. In that case, maybe assign -1 here to make this explicit?

-1 (-EPERM) probably not the best choice. The verifier already has knowledge that a particular tracked map is an inner map or not. So keeping the original error code (mostly -EINVAL) is preferred I think.

Quoted 6 lines
Quoted 3 lines

Ah, I actually missed the fact that verifier actually checks those values (so it's not just >= 0 or < 0), so yeah, let's just pass through. Btw, the value when there is no spinlock is actually -ENOENT.

If there is no BTF, it will be -EINVAL. If there is BTF and no spinlock member, mostly -ENOENT.

You are right, I stand corrected. In both cases the effect should be the same (no way to use spinlock).

Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 7 lines

Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?

Not exactly sure about the reason. Maybe with this patch, it can get proper support. Not 100% sure.

Quoted 8 lines
Quoted 6 lines

No, it won't, because bpf_map_meta_alloc explicitly tests for it:

         if (map_value_has_spin_lock(inner_map)) {
                 fdput(f);
                 return ERR_PTR(-ENOTSUPP);
         }

I mean that this can be removed after my patch and it may work :-)

Ah, got it, yeah, maybe.

Quoted 13 lines
Quoted 13 lines

Maybe Alexei can clarify?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/bpf/map_in_map.c +0 -1
          /* Misc members not needed in bpf_map_meta_equal() check. */
          inner_map_meta->ops = inner_map->ops;
--

2.17.1

#7 lore raw
In reply to Andrii Nakryiko
Quoted 8 lines · Andrii Nakryiko

No, it won't, because bpf_map_meta_alloc explicitly tests for it:

         if (map_value_has_spin_lock(inner_map)) {
                 fdput(f);
                 return ERR_PTR(-ENOTSUPP);
         }

Maybe Alexei can clarify?

inner map can be replaced while progs are running. I don't think added complexity is worth it. Hence it's disallowed for time being.

#8 lore raw

PATCH 2/2 tools/bpf: selftests: add map lookup to test_map_in_map bpf prog

In reply to Yonghong Song

The bpf_map_lookup_elem is added in the bpf program. Without previous patch, the test change will trigger the following error:

  $ ./test_maps
  ...
  ; value_p = bpf_map_lookup_elem(map, &key);
  20: (bf) r1 = r7
  21: (bf) r2 = r8
  22: (85) call bpf_map_lookup_elem#1
  ; if (!value_p || *value_p != 123)
  23: (15) if r0 == 0x0 goto pc+16
   R0=map_value(id=2,off=0,ks=4,vs=4,imm=0) R6=inv1 R7=map_ptr(id=0,off=0,ks=4,vs=4,imm=0)
   R8=fp-8,call_-1 R10=fp0,call_-1 fp-8=mmmmmmmm
  ; if (!value_p || *value_p != 123)
  24: (61) r1 = *(u32 *)(r0 +0)
   R0=map_value(id=2,off=0,ks=4,vs=4,imm=0) R6=inv1 R7=map_ptr(id=0,off=0,ks=4,vs=4,imm=0)
   R8=fp-8,call_-1 R10=fp0,call_-1 fp-8=mmmmmmmm
  bpf_spin_lock cannot be accessed directly by load/store

With the kernel fix in the previous commit, the error goes away.

---
 tools/testing/selftests/bpf/progs/test_map_in_map.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/progs/test_map_in_map.c +4 -0
index ce923e67e08e..2985f262846e 100644
@@ -27,6 +27,7 @@ SEC("xdp_mimtest")
 int xdp_mimtest0(struct xdp_md *ctx)
 {
 	int value = 123;
+	int *value_p;
 	int key = 0;
 	void *map;
 
@@ -35,6 +36,9 @@ int xdp_mimtest0(struct xdp_md *ctx)
 		return XDP_DROP;
 
 	bpf_map_update_elem(map, &key, &value, 0);
+	value_p = bpf_map_lookup_elem(map, &key);
+	if (!value_p || *value_p != 123)
+		return XDP_DROP;
 
 	map = bpf_map_lookup_elem(&mim_hash, &key);
 	if (!map)
Signature · 1 line
2.17.1
Acked-by on v1 2/2
#9 lore raw

Re: [PATCH bpf-next 2/2] tools/bpf: selftests: add map lookup to test_map_in_map bpf prog

Quoted 50 lines · Yonghong Song

The bpf_map_lookup_elem is added in the bpf program. Without previous patch, the test change will trigger the following error:

  $ ./test_maps
  ...
  ; value_p = bpf_map_lookup_elem(map, &key);
  20: (bf) r1 = r7
  21: (bf) r2 = r8
  22: (85) call bpf_map_lookup_elem#1
  ; if (!value_p || *value_p != 123)
  23: (15) if r0 == 0x0 goto pc+16
   R0=map_value(id=2,off=0,ks=4,vs=4,imm=0) R6=inv1 R7=map_ptr(id=0,off=0,ks=4,vs=4,imm=0)
   R8=fp-8,call_-1 R10=fp0,call_-1 fp-8=mmmmmmmm
  ; if (!value_p || *value_p != 123)
  24: (61) r1 = *(u32 *)(r0 +0)
   R0=map_value(id=2,off=0,ks=4,vs=4,imm=0) R6=inv1 R7=map_ptr(id=0,off=0,ks=4,vs=4,imm=0)
   R8=fp-8,call_-1 R10=fp0,call_-1 fp-8=mmmmmmmm
  bpf_spin_lock cannot be accessed directly by load/store

With the kernel fix in the previous commit, the error goes away.

---
 tools/testing/selftests/bpf/progs/test_map_in_map.c | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/progs/test_map_in_map.c +4 -1
index ce923e67e08e..2985f262846e 100644
@@ -27,6 +27,7 @@ SEC("xdp_mimtest")
 int xdp_mimtest0(struct xdp_md *ctx)
 {
        int value = 123;
+       int *value_p;
        int key = 0;
        void *map;
 
@@ -35,6 +36,9 @@ int xdp_mimtest0(struct xdp_md *ctx)
                return XDP_DROP;
 
        bpf_map_update_elem(map, &key, &value, 0);
+       value_p = bpf_map_lookup_elem(map, &key);
+       if (!value_p || *value_p != 123)
+               return XDP_DROP;
 
        map = bpf_map_lookup_elem(&mim_hash, &key);
        if (!map)
--
Quoted 2 lines

2.17.1

LGTM.

#10 lore raw
In reply to Yonghong Song
Quoted 5 lines · Yonghong Song

The inner_map_meta->spin_lock_off is not set correctly during map creation for BPF_MAP_TYPE_ARRAY_OF_MAPS and BPF_MAP_TYPE_HASH_OF_MAPS. This may lead verifier error due to misinformation.
This patch set fixed the issue with Patch #1 for the kernel change and Patch #2 for enhanced selftest test_maps.

Applied, Thanks