bpf: set inner_map_meta->spin_lock_off correctly
Subsystems (5)
cover bpf: set inner_map_meta->spin_lock_off correctly
The inner_map_meta->spin_lock_off is not set correctly during map creation for BPF_MAP_TYPE_ARRAY_OF_MAPS and BPF_MAP_TYPE_HASH_OF_MAPS. This may lead verifier error due to misinformation.
This patch set fixed the issue with Patch #1 for the kernel change and Patch #2 for enhanced selftest test_maps.
Yonghong Song (2):
bpf: set inner_map_meta->spin_lock_off correctly tools/bpf: selftests: add map lookup to test_map_in_map bpf prog
kernel/bpf/map_in_map.c | 1 + tools/testing/selftests/bpf/progs/test_map_in_map.c | 4 ++++ 2 files changed, 5 insertions(+)
Signature · 1 line
2.17.1
PATCH 1/2 bpf: set inner_map_meta->spin_lock_off correctly
Commit d83525ca62cf ("bpf: introduce bpf_spin_lock") introduced bpf_spin_lock and the field spin_lock_off in kernel internal structure bpf_map has the following meaning:
Quoted 1 line
=0 valid offset, <0 error
For every map created, the kernel will ensure spin_lock_off has correct value.
Currently, bpf_map->spin_lock_off is not copied from the inner map to the map_in_map inner_map_meta during a map_in_map type map creation, so inner_map_meta->spin_lock_off = 0.
This will give verifier wrong information that inner_map has bpf_spin_lock and the bpf_spin_lock is defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lock cannot be accessed directly by load/store
This patch fixed the issue by copy inner map's spin_lock_off value to inner_map_meta->spin_lock_off.
--- kernel/bpf/map_in_map.c | 1 + 1 file changed, 1 insertion(+)
kernel/bpf/map_in_map.c +1 -0
@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->value_size = inner_map->value_size; inner_map_meta->map_flags = inner_map->map_flags; inner_map_meta->max_entries = inner_map->max_entries; + inner_map_meta->spin_lock_off = inner_map->spin_lock_off;/* Misc members not needed in bpf_map_meta_equal() check. */ inner_map_meta->ops = inner_map->ops;
Signature · 1 line
2.17.1
Quoted 34 lines · Yonghong Song
Commit
d83525ca62cf("bpf: introducebpf_spin_lock") introducedbpf_spin_lockand the fieldspin_lock_offin kernel internal structurebpf_maphas the following meaning:Quoted 1 line
=0 valid offset, <0 error
For every map created, the kernel will ensure
spin_lock_offhas correct value.Currently,
bpf_map->spin_lock_offis not copied from the inner map to themap_in_mapinner_map_metaduring amap_in_maptype map creation, soinner_map_meta->spin_lock_off= 0.
This will give verifier wrong information thatinner_maphasbpf_spin_lockand thebpf_spin_lockis defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lockcannot be accessed directly by load/storeThis patch fixed the issue by copy inner map's
spin_lock_offvalue toinner_map_meta->spin_lock_off.Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")Signed-off-by Yonghong Song <[email protected]>--- kernel/bpf/map_in_map.c | 1 + 1 file changed, 1 insertion(+)kernel/bpf/map_in_map.c +0
-0
kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61
Quoted 4 lines
kernel/bpf/map_in_map.c +1
-0@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->value_size = inner_map->value_size; inner_map_meta->map_flags = inner_map->map_flags; inner_map_meta->max_entries = inner_map->max_entries; + inner_map_meta->spin_lock_off = inner_map->spin_lock_off;
Looks like spinlock inside inner map is not supported: there is specific check few lines above returning -ENOSUPP for such case. In that case, maybe assign -1 here to make this explicit?
Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?
Quoted 6 lines
kernel/bpf/map_in_map.c +0
-1/* Misc members not needed in bpf_map_meta_equal() check. */ inner_map_meta->ops = inner_map->ops; --2.17.1
Quoted 30 lines · Andrii Nakryiko
Quoted 29 lines · Yonghong Song
Commit
d83525ca62cf("bpf: introducebpf_spin_lock") introducedbpf_spin_lockand the fieldspin_lock_offin kernel internal structurebpf_maphas the following meaning:Quoted 1 line
=0 valid offset, <0 error
For every map created, the kernel will ensure
spin_lock_offhas correct value.Currently,
bpf_map->spin_lock_offis not copied from the inner map to themap_in_mapinner_map_metaduring amap_in_maptype map creation, soinner_map_meta->spin_lock_off= 0.
This will give verifier wrong information thatinner_maphasbpf_spin_lockand thebpf_spin_lockis defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lockcannot be accessed directly by load/storeThis patch fixed the issue by copy inner map's
spin_lock_offvalue toinner_map_meta->spin_lock_off.Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")Signed-off-by Yonghong Song <[email protected]>--- kernel/bpf/map_in_map.c | 1 + 1 file changed, 1 insertion(+)
Quoted 13 lines
kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61
Quoted 9 lines
kernel/bpf/map_in_map.c +1
-0@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->value_size = inner_map->value_size; inner_map_meta->map_flags = inner_map->map_flags; inner_map_meta->max_entries = inner_map->max_entries; + inner_map_meta->spin_lock_off = inner_map->spin_lock_off;Looks like spinlock inside inner map is not supported: there is specific check few lines above returning
-ENOSUPPfor such case. In that case, maybe assign -1 here to make this explicit?
-1 (-EPERM) probably not the best choice. The verifier already has knowledge that a particular tracked map is an inner map or not. So keeping the original error code (mostly -EINVAL) is preferred I think.
Quoted 4 lines
Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?
Not exactly sure about the reason. Maybe with this patch, it can get proper support. Not 100% sure.
Quoted 7 lines
Quoted 7 lines
kernel/bpf/map_in_map.c +0
-1/* Misc members not needed in bpf_map_meta_equal() check. */ inner_map_meta->ops = inner_map->ops; --2.17.1
Andrii Nakryiko
Acked-byQuoted 25 lines · Yonghong Song
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Yonghong Song
Commit
d83525ca62cf("bpf: introducebpf_spin_lock") introducedbpf_spin_lockand the fieldspin_lock_offin kernel internal structurebpf_maphas the following meaning:Quoted 1 line
=0 valid offset, <0 error
For every map created, the kernel will ensure
spin_lock_offhas correct value.Currently,
bpf_map->spin_lock_offis not copied from the inner map to themap_in_mapinner_map_metaduring amap_in_maptype map creation, soinner_map_meta->spin_lock_off= 0.
This will give verifier wrong information thatinner_maphasbpf_spin_lockand thebpf_spin_lockis defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lockcannot be accessed directly by load/store
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
This patch fixed the issue by copy inner map's
spin_lock_offvalue toinner_map_meta->spin_lock_off.Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")Signed-off-by Yonghong Song <[email protected]>
Quoted 18 lines
Quoted 4 lines
Quoted 4 lines
--- kernel/bpf/map_in_map.c | 1 + 1 file changed, 1 insertion(+)Quoted 13 lines
kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61
Quoted 9 lines
kernel/bpf/map_in_map.c +1
-0@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->value_size = inner_map->value_size; inner_map_meta->map_flags = inner_map->map_flags; inner_map_meta->max_entries = inner_map->max_entries; + inner_map_meta->spin_lock_off = inner_map->spin_lock_off;Looks like spinlock inside inner map is not supported: there is specific check few lines above returning
-ENOSUPPfor such case. In that case, maybe assign -1 here to make this explicit?
Quoted 3 lines
-1 (
-EPERM) probably not the best choice. The verifier already has knowledge that a particular tracked map is an inner map or not. So keeping the original error code (mostly-EINVAL) is preferred I think.
Ah, I actually missed the fact that verifier actually checks those values (so it's not just >= 0 or < 0), so yeah, let's just pass through. Btw, the value when there is no spinlock is actually -ENOENT.
Quoted 8 lines
Quoted 5 lines
Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?
Not exactly sure about the reason. Maybe with this patch, it can get proper support. Not 100% sure.
No, it won't, because bpf_map_meta_alloc explicitly tests for it:
if (map_value_has_spin_lock(inner_map)) {
fdput(f);
return ERR_PTR(-ENOTSUPP);
}Maybe Alexei can clarify?
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/bpf/map_in_map.c +0
-1/* Misc members not needed in bpf_map_meta_equal() check. */ inner_map_meta->ops = inner_map->ops; --2.17.1
Quoted 52 lines · Andrii Nakryiko
Quoted 25 lines · Yonghong Song
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Yonghong Song
Commit
d83525ca62cf("bpf: introducebpf_spin_lock") introducedbpf_spin_lockand the fieldspin_lock_offin kernel internal structurebpf_maphas the following meaning:Quoted 1 line
=0 valid offset, <0 error
For every map created, the kernel will ensure
spin_lock_offhas correct value.Currently,
bpf_map->spin_lock_offis not copied from the inner map to themap_in_mapinner_map_metaduring amap_in_maptype map creation, soinner_map_meta->spin_lock_off= 0.
This will give verifier wrong information thatinner_maphasbpf_spin_lockand thebpf_spin_lockis defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lockcannot be accessed directly by load/storeQuoted 5 lines
Quoted 5 lines
Quoted 5 lines
This patch fixed the issue by copy inner map's
spin_lock_offvalue toinner_map_meta->spin_lock_off.Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")Signed-off-by Yonghong Song <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Quoted 18 lines
Quoted 17 lines
kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61
Quoted 13 lines
--- kernel/bpf/map_in_map.c | 1 + 1 file changed, 1 insertion(+)kernel/bpf/map_in_map.c +1
-0@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->value_size = inner_map->value_size; inner_map_meta->map_flags = inner_map->map_flags; inner_map_meta->max_entries = inner_map->max_entries; + inner_map_meta->spin_lock_off = inner_map->spin_lock_off;Looks like spinlock inside inner map is not supported: there is specific check few lines above returning
-ENOSUPPfor such case. In that case, maybe assign -1 here to make this explicit?
Quoted 7 lines
Quoted 3 lines
-1 (
-EPERM) probably not the best choice. The verifier already has knowledge that a particular tracked map is an inner map or not. So keeping the original error code (mostly-EINVAL) is preferred I think.Ah, I actually missed the fact that verifier actually checks those values (so it's not just >= 0 or < 0), so yeah, let's just pass through. Btw, the value when there is no spinlock is actually
-ENOENT.
If there is no BTF, it will be -EINVAL. If there is BTF and no spinlock member, mostly -ENOENT.
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?
Not exactly sure about the reason. Maybe with this patch, it can get proper support. Not 100% sure.
Quoted 6 lines
No, it won't, because
bpf_map_meta_allocexplicitly tests for it:if (map_value_has_spin_lock(inner_map)) { fdput(f); return ERR_PTR(-ENOTSUPP); }
I mean that this can be removed after my patch and it may work :-)
Quoted 12 lines
Maybe Alexei can clarify?
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/bpf/map_in_map.c +0
-1/* Misc members not needed in bpf_map_meta_equal() check. */ inner_map_meta->ops = inner_map->ops; --2.17.1
Quoted 60 lines · Yonghong Song
Quoted 56 lines · Andrii Nakryiko
Quoted 25 lines · Yonghong Song
Quoted 21 lines · Andrii Nakryiko
Quoted 20 lines · Yonghong Song
Commit
d83525ca62cf("bpf: introducebpf_spin_lock") introducedbpf_spin_lockand the fieldspin_lock_offin kernel internal structurebpf_maphas the following meaning:Quoted 1 line
=0 valid offset, <0 error
For every map created, the kernel will ensure
spin_lock_offhas correct value.Currently,
bpf_map->spin_lock_offis not copied from the inner map to themap_in_mapinner_map_metaduring amap_in_maptype map creation, soinner_map_meta->spin_lock_off= 0.
This will give verifier wrong information thatinner_maphasbpf_spin_lockand thebpf_spin_lockis defined at offset 0. An access to offset 0
of a value pointer will trigger the following error:
bpf_spin_lockcannot be accessed directly by load/storeQuoted 5 lines
Quoted 5 lines
Quoted 5 lines
This patch fixed the issue by copy inner map's
spin_lock_offvalue toinner_map_meta->spin_lock_off.Fixes d83525ca62cf ("bpf: introduce bpf_spin_lock")Signed-off-by Yonghong Song <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Quoted 21 lines
Quoted 4 lines
Quoted 4 lines
--- kernel/bpf/map_in_map.c | 1 + 1 file changed, 1 insertion(+)Quoted 13 lines
kernel/bpf/map_in_map.c struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) · line 61
Quoted 9 lines
kernel/bpf/map_in_map.c +1
-0@@ -58,6 +58,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->value_size = inner_map->value_size; inner_map_meta->map_flags = inner_map->map_flags; inner_map_meta->max_entries = inner_map->max_entries; + inner_map_meta->spin_lock_off = inner_map->spin_lock_off;Looks like spinlock inside inner map is not supported: there is specific check few lines above returning
-ENOSUPPfor such case. In that case, maybe assign -1 here to make this explicit?-1 (
-EPERM) probably not the best choice. The verifier already has knowledge that a particular tracked map is an inner map or not. So keeping the original error code (mostly-EINVAL) is preferred I think.
Quoted 6 lines
Quoted 3 lines
Ah, I actually missed the fact that verifier actually checks those values (so it's not just >= 0 or < 0), so yeah, let's just pass through. Btw, the value when there is no spinlock is actually
-ENOENT.If there is no BTF, it will be
-EINVAL. If there is BTF and no spinlock member, mostly-ENOENT.
You are right, I stand corrected. In both cases the effect should be the same (no way to use spinlock).
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 7 lines
Though I guess that also brings up the question: is there any harm in supporting spin lock for inner map and why it was disabled in the first place?
Not exactly sure about the reason. Maybe with this patch, it can get proper support. Not 100% sure.
Quoted 8 lines
Quoted 6 lines
No, it won't, because
bpf_map_meta_allocexplicitly tests for it:if (map_value_has_spin_lock(inner_map)) { fdput(f); return ERR_PTR(-ENOTSUPP); }I mean that this can be removed after my patch and it may work :-)
Ah, got it, yeah, maybe.
Quoted 13 lines
Quoted 13 lines
Maybe Alexei can clarify?
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/bpf/map_in_map.c +0
-1/* Misc members not needed in bpf_map_meta_equal() check. */ inner_map_meta->ops = inner_map->ops; --2.17.1
Quoted 8 lines · Andrii Nakryiko
No, it won't, because
bpf_map_meta_allocexplicitly tests for it:if (map_value_has_spin_lock(inner_map)) { fdput(f); return ERR_PTR(-ENOTSUPP); }Maybe Alexei can clarify?
inner map can be replaced while progs are running. I don't think added complexity is worth it. Hence it's disallowed for time being.
PATCH 2/2 tools/bpf: selftests: add map lookup to test_map_in_map bpf prog
In reply to Yonghong SongThe bpf_map_lookup_elem is added in the bpf program. Without previous patch, the test change will trigger the following error:
$ ./test_maps ... ; value_p = bpf_map_lookup_elem(map, &key); 20: (bf) r1 = r7 21: (bf) r2 = r8 22: (85) call bpf_map_lookup_elem#1 ; if (!value_p || *value_p != 123) 23: (15) if r0 == 0x0 goto pc+16 R0=map_value(id=2,off=0,ks=4,vs=4,imm=0) R6=inv1 R7=map_ptr(id=0,off=0,ks=4,vs=4,imm=0) R8=fp-8,call_-1 R10=fp0,call_-1 fp-8=mmmmmmmm ; if (!value_p || *value_p != 123) 24: (61) r1 = *(u32 *)(r0 +0) R0=map_value(id=2,off=0,ks=4,vs=4,imm=0) R6=inv1 R7=map_ptr(id=0,off=0,ks=4,vs=4,imm=0) R8=fp-8,call_-1 R10=fp0,call_-1 fp-8=mmmmmmmm bpf_spin_lock cannot be accessed directly by load/store
With the kernel fix in the previous commit, the error goes away.
--- tools/testing/selftests/bpf/progs/test_map_in_map.c | 4 ++++ 1 file changed, 4 insertions(+)
tools/testing/selftests/bpf/progs/test_map_in_map.c +4 -0
@@ -27,6 +27,7 @@ SEC("xdp_mimtest")int xdp_mimtest0 (struct xdp_md *ctx) {int value =123 ; +int *value_p;int key =0 ;void *map;
@@ -35,6 +36,9 @@ int xdp_mimtest0(struct xdp_md *ctx)return XDP_DROP ;bpf_map_update_elem (map, &key, &value,0 ); + value_p =bpf_map_lookup_elem (map, &key); +if (!value_p || *value_p !=123 ) +return XDP_DROP ; map =bpf_map_lookup_elem (&mim_hash, &key);if (!map)
Signature · 1 line
2.17.1
Re: [PATCH bpf-next 2/2] tools/bpf: selftests: add map lookup to test_map_in_map bpf prog
Quoted 50 lines · Yonghong Song
The
bpf_map_lookup_elemis added in the bpf program. Without previous patch, the test change will trigger the following error:$ ./test_maps ... ; value_p = bpf_map_lookup_elem(map, &key); 20: (bf) r1 = r7 21: (bf) r2 = r8 22: (85) call bpf_map_lookup_elem#1 ; if (!value_p || *value_p != 123) 23: (15) if r0 == 0x0 goto pc+16 R0=map_value(id=2,off=0,ks=4,vs=4,imm=0) R6=inv1 R7=map_ptr(id=0,off=0,ks=4,vs=4,imm=0) R8=fp-8,call_-1 R10=fp0,call_-1 fp-8=mmmmmmmm ; if (!value_p || *value_p != 123) 24: (61) r1 = *(u32 *)(r0 +0) R0=map_value(id=2,off=0,ks=4,vs=4,imm=0) R6=inv1 R7=map_ptr(id=0,off=0,ks=4,vs=4,imm=0) R8=fp-8,call_-1 R10=fp0,call_-1 fp-8=mmmmmmmm bpf_spin_lock cannot be accessed directly by load/storeWith the kernel fix in the previous commit, the error goes away.
Signed-off-by Yonghong Song <[email protected]>--- tools/testing/selftests/bpf/progs/test_map_in_map.c | 4 ++++ 1 file changed, 4 insertions(+)tools/testing/selftests/bpf/progs/test_map_in_map.c +4
-1@@ -27,6 +27,7 @@ SEC("xdp_mimtest")int xdp_mimtest0 (struct xdp_md *ctx) {int value =123 ; +int *value_p;int key =0 ;void *map;@@ -35,6 +36,9 @@ int xdp_mimtest0(struct xdp_md *ctx)return XDP_DROP ;bpf_map_update_elem (map, &key, &value,0 ); + value_p =bpf_map_lookup_elem (map, &key); +if (!value_p || *value_p !=123 ) +return XDP_DROP ; map =bpf_map_lookup_elem (&mim_hash, &key);if (!map) --
Quoted 2 lines
2.17.1
LGTM.
Alexei Starovoitov
AppliedQuoted 5 lines · Yonghong Song
The
inner_map_meta->spin_lock_offis not set correctly during map creation forBPF_MAP_TYPE_ARRAY_OF_MAPSandBPF_MAP_TYPE_HASH_OF_MAPS. This may lead verifier error due to misinformation.
This patch set fixed the issue with Patch #1 for the kernel change and Patch #2 for enhanced selftesttest_maps.
Applied, Thanks