mempool: fix memory allocation in memzones during retry.
PATCH 1/1 mempool: fix memory allocation in memzones during retry.
If allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of mem_size and it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of the mem_size.
It is wrong that max_alloc_size is divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.
--- lib/librte_mempool/rte_mempool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_mempool/rte_mempool.c +1 -1
@@ -635,7 +635,7 @@ struct pagesz_walk_arg {RTE_MIN ((size_t )mem_size, max_alloc_size), mp->socket_id, mz_flags, align); -if (mz ==NULL && rte_errno !=ENOMEM ) +if ((mz !=NULL ) || (mz ==NULL && rte_errno !=ENOMEM ))break ; max_alloc_size = RTE_MIN(max_alloc_size,
Signature · 1 line
1.8.3.1
Quoted 10 lines · Zhike Wang
If allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of
mem_sizeand it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of themem_size.
Quoted 7 lines
It is wrong that
max_alloc_sizeis divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.Signed-off-by Andrew Rybchenko <[email protected]>Signed-off-by Zhike Wang <[email protected]>
Quoted 18 lines
--- lib/librte_mempool/rte_mempool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/librte_mempool/rte_mempool.c +1
-1@@ -635,7 +635,7 @@ struct pagesz_walk_arg {RTE_MIN ((size_t )mem_size, max_alloc_size), mp->socket_id, mz_flags, align); -if (mz ==NULL && rte_errno !=ENOMEM ) +if ((mz !=NULL ) || (mz ==NULL && rte_errno !=ENOMEM ))break ; max_alloc_size = RTE_MIN(max_alloc_size,
Quoted 26 lines · Zhike Wang
If allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of
mem_sizeand it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of themem_size.It is wrong that
max_alloc_sizeis divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.Signed-off-by Andrew Rybchenko <[email protected]>Signed-off-by Zhike Wang <[email protected]>--- lib/librte_mempool/rte_mempool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/librte_mempool/rte_mempool.c +0
-0
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg { · line 638
Quoted 5 lines
lib/librte_mempool/rte_mempool.c +1
-1@@ -635,7 +635,7 @@ struct pagesz_walk_arg {RTE_MIN ((size_t )mem_size, max_alloc_size), mp->socket_id, mz_flags, align); -if (mz ==NULL && rte_errno !=ENOMEM ) +if ((mz !=NULL ) || (mz ==NULL && rte_errno !=ENOMEM ))
I think checking mz == NULL for the second time is redundant, as if we're hitting the second branch, we've already failed the "mz != NULL" test and can therefore assume that mz == NULL.
That said, i'm struggling to think of circumstances where this would matter. Could you please provide an example?
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg { · line 641
Quoted 4 lines
lib/librte_mempool/rte_mempool.c +0
-0break ; max_alloc_size = RTE_MIN(max_alloc_size,
This should have a Fixes: tag.
Signature · 2 lines
Thanks, Anatoly
Quoted 22 lines · Anatoly
Quoted 21 lines · Zhike Wang
If allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of
mem_sizeand it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of themem_size.It is wrong that
max_alloc_sizeis divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.Signed-off-by Andrew Rybchenko <[email protected]>Signed-off-by Zhike Wang <[email protected]>---
lib/librte_mempool/rte_mempool.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg { · line 638
Quoted 10 lines
lib/librte_mempool/rte_mempool.c +1
-0@@ -635,7 +635,7 @@ struct pagesz_walk_arg { RTE_MIN((size_t)mem_size, max_alloc_size), mp->socket_id, mz_flags, align); - if (mz == NULL && rte_errno != ENOMEM) + if ((mz != NULL) || (mz == NULL && rte_errno != ENOMEM))I think checking mz == NULL for the second time is redundant, as if we're hitting the second branch, we've already failed the "mz != NULL" test and can therefore assume that mz == NULL.
Yes, of course. (Also parenthesis will be not required.)
Quoted 3 lines
That said, i'm struggling to think of circumstances where this would matter. Could you please provide an example?
If the question about break in the case of mz != NULL, it is important to avoid decreasing max_alloc_size to try the same size once again if one more iteration is needed to allocate remaining elements.
Quoted 7 lines
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg {
Quoted 4 lines
lib/librte_mempool/rte_mempool.c +0
-0break; max_alloc_size = RTE_MIN(max_alloc_size,This should have a Fixes: tag.
Yes, missed it.
Many thanks for the review.
Quoted 41 lines · Andrew Rybchenko
Quoted 22 lines · Anatoly
Quoted 21 lines · Zhike Wang
If allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of
mem_sizeand it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of themem_size.It is wrong that
max_alloc_sizeis divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.Signed-off-by Andrew Rybchenko <[email protected]>Signed-off-by Zhike Wang <[email protected]>--- lib/librte_mempool/rte_mempool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 14 lines
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg { · line 638
Quoted 10 lines
lib/librte_mempool/rte_mempool.c +1
-0@@ -635,7 +635,7 @@ struct pagesz_walk_arg { RTE_MIN((size_t)mem_size, max_alloc_size), mp->socket_id, mz_flags, align); - if (mz == NULL && rte_errno != ENOMEM) + if ((mz != NULL) || (mz == NULL && rte_errno != ENOMEM))I think checking mz == NULL for the second time is redundant, as if we're hitting the second branch, we've already failed the "mz != NULL" test and can therefore assume that mz == NULL.
Yes, of course. (Also parenthesis will be not required.)
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
That said, i'm struggling to think of circumstances where this would matter. Could you please provide an example?
If the question about break in the case of mz != NULL, it is important to avoid decreasing
max_alloc_sizeto try the same size once again if one more iteration is needed to allocate remaining elements.
Right, no further questions :)
Quoted 13 lines
Quoted 8 lines
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg {
Quoted 5 lines
lib/librte_mempool/rte_mempool.c +0
-0break; max_alloc_size = RTE_MIN(max_alloc_size,This should have a Fixes: tag.
Yes, missed it.
Many thanks for the review.
Signature · 2 lines
Thanks, Anatoly
Thanks for review.
Sent out v2.
Br,
Zhike Wang
JDCloud, Product Development, IaaS
------------------------------------------------------------------------------------------------
Mobile/+86 13466719566
E- mail/[email protected]
Address/5F Building A,North-Star Century Center,8 Beichen West Street,Chaoyang District Beijing Https://JDCloud.com
------------------------------------------------------------------------------------------------
-----Original Message-----
From: Burakov, Anatoly [mailto:[email protected]]
Sent: Monday, July 13, 2020 10:53 PM
To: Andrew Rybchenko; 王志克; [email protected]
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH] mempool: fix memory allocation in memzones during retry.
Quoted 41 lines · Andrew Rybchenko
Quoted 22 lines · Anatoly
Quoted 21 lines · Zhike Wang
If allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of
mem_sizeand it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of themem_size.It is wrong that
max_alloc_sizeis divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.Signed-off-by Andrew Rybchenko <[email protected]>Signed-off-by Zhike Wang <[email protected]>--- lib/librte_mempool/rte_mempool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 14 lines
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg { · line 638
Quoted 10 lines
lib/librte_mempool/rte_mempool.c +1
-0@@ -635,7 +635,7 @@ struct pagesz_walk_arg { RTE_MIN((size_t)mem_size, max_alloc_size), mp->socket_id, mz_flags, align); - if (mz == NULL && rte_errno != ENOMEM) + if ((mz != NULL) || (mz == NULL && rte_errno != ENOMEM))I think checking mz == NULL for the second time is redundant, as if we're hitting the second branch, we've already failed the "mz != NULL" test and can therefore assume that mz == NULL.
Yes, of course. (Also parenthesis will be not required.)
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
That said, i'm struggling to think of circumstances where this would matter. Could you please provide an example?
If the question about break in the case of mz != NULL, it is important to avoid decreasing
max_alloc_sizeto try the same size once again if one more iteration is needed to allocate remaining elements.
Right, no further questions :)
Quoted 13 lines
Quoted 8 lines
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg {
Quoted 5 lines
lib/librte_mempool/rte_mempool.c +0
-0break; max_alloc_size = RTE_MIN(max_alloc_size,This should have a Fixes: tag.
Yes, missed it.
Many thanks for the review.
Signature · 2 lines
Thanks, Anatoly
PATCH v2 1/1 mempool: fix memory allocation in memzones during retry.
In reply to Zhike WangIf allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of mem_size and it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of the mem_size.
It is wrong that max_alloc_size is divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.
--- lib/librte_mempool/rte_mempool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_mempool/rte_mempool.c +1 -1
@@ -635,7 +635,7 @@ struct pagesz_walk_arg {RTE_MIN ((size_t )mem_size, max_alloc_size), mp->socket_id, mz_flags, align); -if (mz ==NULL && rte_errno !=ENOMEM ) +if (mz !=NULL || rte_errno !=ENOMEM )break ; max_alloc_size = RTE_MIN(max_alloc_size,
Signature · 1 line
1.8.3.1
Quoted 29 lines · Zhike Wang
If allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of
mem_sizeand it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of themem_size.It is wrong that
max_alloc_sizeis divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.Fixes 3a3d0c75b43e ("mempool: fix slow allocation of large pools")Signed-off-by Andrew Rybchenko <[email protected]>Signed-off-by Zhike Wang <[email protected]>--- lib/librte_mempool/rte_mempool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/librte_mempool/rte_mempool.c +0
-0
lib/librte_mempool/rte_mempool.c struct pagesz_walk_arg { · line 641
Quoted 8 lines
lib/librte_mempool/rte_mempool.c +1
-1@@ -635,7 +635,7 @@ struct pagesz_walk_arg {RTE_MIN ((size_t )mem_size, max_alloc_size), mp->socket_id, mz_flags, align); -if (mz ==NULL && rte_errno !=ENOMEM ) +if (mz !=NULL || rte_errno !=ENOMEM )break ; max_alloc_size = RTE_MIN(max_alloc_size,
Signature · 2 lines
Thanks, Anatoly
Thomas Monjalon
Applied14/07/2020 11:32, Burakov, Anatoly:
Quoted 11 lines · Zhike Wang
Quoted 10 lines
If allocation is successful on the first attempt, typically there is no problem since we allocated everything required and we'll terminate the loop (if memory chunk is really sufficient to populate required number of mempool elements).
If the first attempt fails, we try to allocate half of
mem_sizeand it succeed, we'll have one more iteration of the for-loop to allocate memory for remaining elements and should not try the next time with quarter of themem_size.
Quoted 6 lines
Quoted 6 lines
It is wrong that
max_alloc_sizeis divided by 2 in the case of successful allocation as well, or invalid memory can be allocated, and leads to population failure, then errno other than ENOMEM may be returned.Fixes 3a3d0c75b43e ("mempool: fix slow allocation of large pools")
Quoted 4 lines
Quoted 2 lines
Signed-off-by Andrew Rybchenko <[email protected]>Signed-off-by Zhike Wang <[email protected]>Acked-by Anatoly Burakov <[email protected]>
Applied, thanks