mm/slub: Remove repeated action in calculate_order()
Subsystems (2)
PATCH v3 1/1 mm/slub: Remove repeated action in calculate_order()
To calculate order, calc_slab_order() is called repeatly changing the fract_leftover. Thus, the branch which is not dependent on fract_leftover is executed repeatly. So make it run only once.
Plus, when min_object reached to 1, we set fract_leftover to 1. In this case, we can calculate order by max(slub_min_order, get_order(size)) instead of calling calc_slab_order().
No functional impact expected.
---
mm/slub.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)
mm/slub.c +7 -11
@@ -3795,9 +3795,6 @@ static inline unsigned int calc_slab_order(unsigned int size,unsigned int min_order = slub_min_order;unsigned int order; -if (order_objects (min_order, size) >MAX_OBJS_PER_PAGE ) -return get_order (size *MAX_OBJS_PER_PAGE ) -1 ; -for (order =max (min_order, (unsigned int )get_order (min_objects * size)); order <= max_order; order++) {
@@ -3820,6 +3817,11 @@ static inline int calculate_order(unsigned int size)unsigned int max_objects;unsigned int nr_cpus; +if (unlikely (order_objects (slub_min_order, size) >MAX_OBJS_PER_PAGE )) { + order =get_order (size *MAX_OBJS_PER_PAGE ) -1 ; + goto out; + } +/* * Attempt to find best configuration for a slab. This * works by first attempting to generate a layout with
@@ -3865,14 +3867,8 @@ static inline int calculate_order(unsigned int size)* We were unable to place multiple objects in a slab. Now * lets see if we can place a single object there. */ - order =calc_slab_order (size,1 , slub_max_order,1 ); -if (order <= slub_max_order) -return order; - -/* -* Doh this slab cannot be placed using slub_max_order. -*/ - order =calc_slab_order (size,1 ,MAX_ORDER ,1 ); + order = max_t(unsigned int, slub_min_order, get_order(size)); +out:if (order <MAX_ORDER )return order;return -ENOSYS ;
Signature · 1 line
2.30.2
Quoted 32 lines · Wonhyuk Yang
To calculate order,
calc_slab_order()is called repeatly changing thefract_leftover. Thus, the branch which is not dependent onfract_leftoveris executed repeatly. So make it run only once.Plus, when
min_objectreached to 1, we setfract_leftoverto 1. In this case, we can calculate order bymax(slub_min_order, get_order(size))instead of callingcalc_slab_order().No functional impact expected.
Signed-off-by Wonhyuk Yang <[email protected]>Reviewed-by Hyeonggon Yoo <[email protected]>---
mm/slub.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)mm/slub.c +0
-3@@ -3795,9 +3795,6 @@ static inline unsigned int calc_slab_order(unsigned int size,unsigned int min_order = slub_min_order;unsigned int order; -if (order_objects (min_order, size) >MAX_OBJS_PER_PAGE ) -return get_order (size *MAX_OBJS_PER_PAGE ) -1 ; -for (order =max (min_order, (unsigned int )get_order (min_objects * size)); order <= max_order; order++) {
mm/slub.c static inline int calculate_order(unsigned int size) · line 3823
Quoted 7 lines
mm/slub.c +4
-0@@ -3820,6 +3817,11 @@ static inline int calculate_order(unsigned int size)unsigned int max_objects;unsigned int nr_cpus; +if (unlikely (order_objects (slub_min_order, size) >MAX_OBJS_PER_PAGE )) { + order =get_order (size *MAX_OBJS_PER_PAGE ) -1 ; + goto out; + }
Hm interestingly, both before and after your patch, MAX_OBJS_PER_PAGE might be theoretically overflowed not by slub_min_order, but then with higher orders. Seems to be prevented only as a side-effect of fragmentation close to none, thus higher orders not attempted. Would be maybe less confusing to check that explicitly. Even if that's wasteful, but this is not really perf critical code.
Quoted 7 lines
mm/slub.c +1
-0+/* * Attempt to find best configuration for a slab. This * works by first attempting to generate a layout with @@ -3865,14 +3867,8 @@ static inline int calculate_order(unsigned int size)* We were unable to place multiple objects in a slab. Now * lets see if we can place a single object there.
mm/slub.c static inline int calculate_order(unsigned int size) · line 3870
Quoted 10 lines
mm/slub.c +1
-8*/ - order =calc_slab_order (size,1 , slub_max_order,1 ); -if (order <= slub_max_order) -return order; - -/* -* Doh this slab cannot be placed using slub_max_order. -*/ - order =calc_slab_order (size,1 ,MAX_ORDER ,1 ); + order = max_t(unsigned int, slub_min_order, get_order(size));
If we failed to assign order above, then AFAICS it means even slub_min_order will not give us more than 1 object per slub. Thus it doesn't make sense to use it in a max() formula, and we can just se get_order(), no?
Quoted 4 lines
mm/slub.c +1
-0+out:if (order <MAX_ORDER )return order;return -ENOSYS ;
Quoted 37 lines · Vlastimil Babka
Quoted 35 lines · Wonhyuk Yang
To calculate order,
calc_slab_order()is called repeatly changing thefract_leftover. Thus, the branch which is not dependent onfract_leftoveris executed repeatly. So make it run only once.Plus, when
min_objectreached to 1, we setfract_leftoverto 1. In this case, we can calculate order bymax(slub_min_order, get_order(size))instead of callingcalc_slab_order().No functional impact expected.
Signed-off-by Wonhyuk Yang <[email protected]>Reviewed-by Hyeonggon Yoo <[email protected]>---
mm/slub.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)mm/slub.c +0
-3@@ -3795,9 +3795,6 @@ static inline unsigned int calc_slab_order(unsigned int size,unsigned int min_order = slub_min_order;unsigned int order; -if (order_objects (min_order, size) >MAX_OBJS_PER_PAGE ) -return get_order (size *MAX_OBJS_PER_PAGE ) -1 ; -for (order =max (min_order, (unsigned int )get_order (min_objects * size)); order <= max_order; order++) {@@ -3820,6 +3817,11 @@ static inline int calculate_order(unsigned int size)unsigned int max_objects;unsigned int nr_cpus;
Quoted 11 lines
mm/slub.c static inline int calculate_order(unsigned int size) · line 3823
Quoted 4 lines
mm/slub.c +4
-0+if (unlikely (order_objects (slub_min_order, size) >MAX_OBJS_PER_PAGE )) { + order =get_order (size *MAX_OBJS_PER_PAGE ) -1 ; + goto out; + }Hm interestingly, both before and after your patch,
MAX_OBJS_PER_PAGEmight be theoretically overflowed not byslub_min_order, but then with higher orders. Seems to be prevented only as a side-effect of fragmentation close to none, thus higher orders not attempted. Would be maybe less confusing to check that explicitly. Even if that's wasteful, but this is not really perf critical code.
Yes, I agree that checking the overflow of object number explicitly is better even if
it is almost impossible. But it checked repeatedly by calling calc_slab_order(). It
seems to me that is unnecessary doesn't it?
Quoted 14 lines
Quoted 14 lines
mm/slub.c +1
-5+/* * Attempt to find best configuration for a slab. This * works by first attempting to generate a layout with @@ -3865,14 +3867,8 @@ static inline int calculate_order(unsigned int size)* We were unable to place multiple objects in a slab. Now * lets see if we can place a single object there. */ - order =calc_slab_order (size,1 , slub_max_order,1 ); -if (order <= slub_max_order) -return order; - -/*
Quoted 8 lines
mm/slub.c static inline int calculate_order(unsigned int size) · line 3870
Quoted 4 lines
mm/slub.c +1
-3-* Doh this slab cannot be placed using slub_max_order. -*/ - order =calc_slab_order (size,1 ,MAX_ORDER ,1 ); + order = max_t(unsigned int, slub_min_order, get_order(size));If we failed to assign order above, then AFAICS it means even
slub_min_orderwill not give us more than 1 object per slub. Thus it doesn't make sense to use it in amax()formula, and we can just seget_order(), no?
That's sounds reasonable. When it reached here, we don't need to keep the slub_min_order.
Quoted 6 lines
Quoted 5 lines
mm/slub.c +1
-0+out:if (order <MAX_ORDER )return order;return -ENOSYS ;
Quoted 12 lines · Wonhyuk Yang
Quoted 11 lines
mm/slub.c static inline int calculate_order(unsigned int size) · line 3823
Quoted 4 lines
mm/slub.c +4
-0+if (unlikely (order_objects (slub_min_order, size) >MAX_OBJS_PER_PAGE )) { + order =get_order (size *MAX_OBJS_PER_PAGE ) -1 ; + goto out; + }Hm interestingly, both before and after your patch,
MAX_OBJS_PER_PAGEmight be theoretically overflowed not byslub_min_order, but then with higher orders. Seems to be prevented only as a side-effect of fragmentation close to none, thus higher orders not attempted. Would be maybe less confusing to check that explicitly. Even if that's wasteful, but this is not really perf critical code.
Quoted 5 lines
Yes, I agree that checking the overflow of object number explicitly is better even if
it is almost impossible. But it checked repeatedly by callingcalc_slab_order(). It
seems to me that is unnecessary doesn't it?
Yeah I'm OK with the goal of your patch.