small fixes for mbuf's dynamic field/flag feature
cover small fixes for mbuf's dynamic field/flag feature
This series contains some small fixes and enhancement for mbuf's dynamic field/flag feature.
Xiaolong Ye (5):
mbuf: fix out-of-bounds access at dyn field register
mbuf: fix missing errno for dyn field/flag registration
mbuf: fix free_space setting for dynamic field
mbuf: fix a dynamic field dump log
mbuf: support to dump free_flags for dynamic flag
lib/librte_mbuf/rte_mbuf_dyn.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/5 mbuf: fix out-of-bounds access at dyn field register
We should make sure off + size < sizeof(struct rte_mbuf) to avoid possible out-of-bounds access of free_space array, there is no issue currently due to the low bits of free_flags (which is adjacent to free_space) are always set to 0. But we shouldn't rely on it since it's fragile and layout of struct mbuf_dyn_shm may be changed in the future. This patch adds boundary check explicitly to avoid potential risk of out-of-bounds access.
--- lib/librte_mbuf/rte_mbuf_dyn.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
lib/librte_mbuf/rte_mbuf_dyn.c +2 -1
@@ -69,7 +69,8 @@ process_score(void)for (off =0 ; off <sizeof (struct rte_mbuf ); off++) {/* get the size of the free zone */ -for (size =0 ; shm->free_space[off + size]; size++) +for (size =0 ; (off + size) <sizeof (struct rte_mbuf ) && + shm->free_space[off + size]; size++) ;if (size ==0 )continue ;
Signature · 1 line
2.17.1
PATCH 2/5 mbuf: fix missing errno for dyn field/flag registration
In reply to Xiaolong YeSet rte_errno as ENOMEM when allocation failure.
--- lib/librte_mbuf/rte_mbuf_dyn.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
lib/librte_mbuf/rte_mbuf_dyn.c +8 -2
@@ -278,12 +278,15 @@ __rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield *params, mbuf_dynfield_tailq.head, mbuf_dynfield_list); te =rte_zmalloc ("MBUF_DYNFIELD_TAILQ_ENTRY" ,sizeof (*te),0 ); -if (te ==NULL ) +if (te ==NULL ) { + rte_errno =ENOMEM ;return -1 ; + } mbuf_dynfield =rte_zmalloc ("mbuf_dynfield" ,sizeof (*mbuf_dynfield),0 );if (mbuf_dynfield ==NULL ) {rte_free (te); + rte_errno =ENOMEM ;return -1 ; }
@@ -456,12 +459,15 @@ __rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params, mbuf_dynflag_tailq.head, mbuf_dynflag_list); te =rte_zmalloc ("MBUF_DYNFLAG_TAILQ_ENTRY" ,sizeof (*te),0 ); -if (te ==NULL ) +if (te ==NULL ) { + rte_errno =ENOMEM ;return -1 ; + } mbuf_dynflag =rte_zmalloc ("mbuf_dynflag" ,sizeof (*mbuf_dynflag),0 );if (mbuf_dynflag ==NULL ) {rte_free (te); + rte_errno =ENOMEM ;return -1 ; }
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH 2/5] mbuf: fix missing errno for dyn field/flag registration
Quoted 6 lines · Xiaolong Ye
Set
rte_errnoas ENOMEM when allocation failure.Fixes 4958ca3a443a ("mbuf: support dynamic fields and flags")Signed-off-by Xiaolong Ye <[email protected]>
Good catch, I was wrongly expecting that rte_malloc() functions sets rte_errno on error.
I wonder if the malloc API shouldn't be modified, because I can see several places where the same kind of fix would be needed: rte_efd, rte_cuckoo_hash, rte_fbk_hash, rte_member, ...
Anatoly, what do you think?
PATCH 3/5 mbuf: fix free_space setting for dynamic field
In reply to Xiaolong YeThe value free_space[i] is used to save the size of biggest aligned element that can fit in the zone, current implementation has one flaw, for example, if user registers dynfield1 (size = 4, align = 4, req = 124) first, the free_space would be as below after registration:
0070: 08 08 08 08 08 08 08 08
0078: 08 08 08 08 00 00 00 00
Then if user continues to register dynfield2 (size = 4, align = 4), free_space would become:
0070: 00 00 00 00 04 04 04 04
0078: 04 04 04 04 00 00 00 00
Further request dynfield3 (size = 8, align = 8) would fail to register due to alignment requirement can't be satisfied, though there is enough space remained in mbuf.
This patch fixes above issue by saving alignment only in aligned zone, after the fix, above registrations order can be satisfied, free_space would be like:
After dynfield1 registration:
0070: 08 08 08 08 08 08 08 08
0078: 04 04 04 04 00 00 00 00
After dynfield2 registration:
0070: 08 08 08 08 08 08 08 08
0078: 00 00 00 00 00 00 00 00
After dynfield3 registration:
0070: 00 00 00 00 00 00 00 00
0078: 00 00 00 00 00 00 00 00
This patch also reduces iterations in process_score() by jumping align steps in each loop.
--- lib/librte_mbuf/rte_mbuf_dyn.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
lib/librte_mbuf/rte_mbuf_dyn.c +8 -3
@@ -67,13 +67,16 @@ process_score(void) shm->free_space[i] =1 ; } -for (off =0 ; off <sizeof (struct rte_mbuf ); off++) { + off =0 ; +while (off <sizeof (struct rte_mbuf )) {/* get the size of the free zone */ for (size =0 ; (off + size) <sizeof (struct rte_mbuf ) && shm->free_space[off + size]; size++) ; -if (size ==0 ) +if (size ==0 ) { + off++;continue ; + }/* get the alignment of biggest object that can fit in * the zone at this offset.
@@ -84,8 +87,10 @@ process_score(void) ;/* save it in free_space[] */ -for (i = off; i < off + size; i++) +for (i = off; i < off + align; i++) shm->free_space[i] =RTE_MAX (align, shm->free_space[i]); + + off += align; } }
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH 3/5] mbuf: fix free_space setting for dynamic field
Quoted 37 lines · Xiaolong Ye
The value free_space[i] is used to save the size of biggest aligned element that can fit in the zone, current implementation has one flaw, for example, if user registers dynfield1 (size = 4, align = 4, req = 124) first, the
free_spacewould be as below after registration:0070: 08 08 08 08 08 08 08 08
0078: 08 08 08 08 00 00 00 00Then if user continues to register dynfield2 (size = 4, align = 4),
free_spacewould become:0070: 00 00 00 00 04 04 04 04
0078: 04 04 04 04 00 00 00 00Further request dynfield3 (size = 8, align = 8) would fail to register due to alignment requirement can't be satisfied, though there is enough space remained in mbuf.
This patch fixes above issue by saving alignment only in aligned zone, after the fix, above registrations order can be satisfied,
free_spacewould be like:After dynfield1 registration:
0070: 08 08 08 08 08 08 08 08
0078: 04 04 04 04 00 00 00 00After dynfield2 registration:
0070: 08 08 08 08 08 08 08 08
0078: 00 00 00 00 00 00 00 00After dynfield3 registration:
0070: 00 00 00 00 00 00 00 00
0078: 00 00 00 00 00 00 00 00
Quoted 7 lines
This patch also reduces iterations in
process_score()by jumping align steps in each loop.Fixes 4958ca3a443a ("mbuf: support dynamic fields and flags")Signed-off-by Xiaolong Ye <[email protected]>
Thanks!
PATCH 4/5 mbuf: fix a dynamic field dump log
In reply to Xiaolong YeFor each mbuf byte, free_space[i] == 0 means the space is occupied, free_space[i] != 0 means space is free.
--- lib/librte_mbuf/rte_mbuf_dyn.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_mbuf/rte_mbuf_dyn.c +1 -1
@@ -552,7 +552,7 @@ void rte_mbuf_dyn_dump(FILE *out) dynflag->params.name, dynflag->bitnum, dynflag->params.flags); } -fprintf (out,"Free space in mbuf (0 = free, value = zone alignment):\n" ); +fprintf (out,"Free space in mbuf (0 = occupied, value = free zone alignment):\n" );for (i =0 ; i <sizeof (struct rte_mbuf ); i++) {if ((i %8 ) ==0 )fprintf (out," %4.4zx: " , i);
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH 4/5] mbuf: fix a dynamic field dump log
Quoted 7 lines · Xiaolong Ye
For each mbuf byte, free_space[i] == 0 means the space is occupied, free_space[i] != 0 means space is free.
Fixes 4958ca3a443a ("mbuf: support dynamic fields and flags")Signed-off-by Xiaolong Ye <[email protected]>
PATCH 5/5 mbuf: support to dump free_flags for dynamic flag
In reply to Xiaolong YeAdd support to dump free_flags as below format:
Free bit in mbuf->ol_flags (0 = occupied, 1 = free):
0000: 0 0 0 0 0 0 0 0
0008: 0 0 0 0 0 0 0 0
0010: 0 0 0 0 0 0 0 1
0018: 1 1 1 1 1 1 1 1
0020: 1 1 1 1 1 1 1 1
0028: 1 0 0 0 0 0 0 0
0030: 0 0 0 0 0 0 0 0
0038: 0 0 0 0 0 0 0 0
--- lib/librte_mbuf/rte_mbuf_dyn.c | 8 ++++++++ 1 file changed, 8 insertions(+)
lib/librte_mbuf/rte_mbuf_dyn.c +8 -0
@@ -559,5 +559,13 @@ void rte_mbuf_dyn_dump(FILE *out)fprintf (out,"%2.2x%s" , shm->free_space[i], (i %8 !=7 ) ?" " :"\n" ); } +fprintf (out,"Free bit in mbuf->ol_flags (0 = occupied, 1 = free):\n" ); +for (i =0 ; i <sizeof (uint64_t ) *CHAR_BIT ; i++) { +if ((i %8 ) ==0 ) +fprintf (out," %4.4zx: " , i); +fprintf (out,"%1.1x%s" , (shm->free_flags & (1ULL << i)) ?1 :0 , + (i %8 !=7 ) ?" " :"\n" ); + } +rte_mcfg_tailq_write_unlock (); }
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH 5/5] mbuf: support to dump free_flags for dynamic flag
Quoted 12 lines · Xiaolong Ye
Add support to dump
free_flagsas below format:Free bit in
mbuf->ol_flags(0 = occupied, 1 = free):
0000: 0 0 0 0 0 0 0 0
0008: 0 0 0 0 0 0 0 0
0010: 0 0 0 0 0 0 0 1
0018: 1 1 1 1 1 1 1 1
0020: 1 1 1 1 1 1 1 1
0028: 1 0 0 0 0 0 0 0
0030: 0 0 0 0 0 0 0 0
0038: 0 0 0 0 0 0 0 0
Quoted 1 line
Signed-off-by Xiaolong Ye <[email protected]>
Thomas Monjalon
Applied13/06/2020 17:49, Xiaolong Ye:
Quoted 10 lines
This series contains some small fixes and enhancement for mbuf's dynamic field/flag feature.
Xiaolong Ye (5):
mbuf: fix out-of-bounds access at dyn field register
mbuf: fix missing errno for dyn field/flag registration
mbuf: fixfree_spacesetting for dynamic field
mbuf: fix a dynamic field dump log
mbuf: support to dumpfree_flagsfor dynamic flag
Quoted 2 lines
lib/librte_mbuf/rte_mbuf_dyn.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)
Applied, thanks a lot for the nice fixes