Skip to contentSkip to conversation

mm/slub: Remove repeated action in calculate_order()

PATCH In review Wonhyuk Yang 2 participants 4 messages Apr 30, 2022 – May 9, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 mm/slub: Remove repeated action in calculate_order()

To calculate order, calc_slab_order() is called repeatly changing the fract_leftover. Thus, the branch which is not dependent on fract_leftover is executed repeatly. So make it run only once.

Plus, when min_object reached to 1, we set fract_leftover to 1. In this case, we can calculate order by max(slub_min_order, get_order(size)) instead of calling calc_slab_order().

No functional impact expected.

---

 mm/slub.c | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)
mm/slub.c +7 -11
index ed5c2c03a47a..1fe4d62b72b8 100644
@@ -3795,9 +3795,6 @@ static inline unsigned int calc_slab_order(unsigned int size,
 	unsigned int min_order = slub_min_order;
 	unsigned int order;
 
-	if (order_objects(min_order, size) > MAX_OBJS_PER_PAGE)
-		return get_order(size * MAX_OBJS_PER_PAGE) - 1;
-
 	for (order = max(min_order, (unsigned int)get_order(min_objects * size));
 			order <= max_order; order++) {
 
@@ -3820,6 +3817,11 @@ static inline int calculate_order(unsigned int size)
 	unsigned int max_objects;
 	unsigned int nr_cpus;
 
+	if (unlikely(order_objects(slub_min_order, size) > MAX_OBJS_PER_PAGE)) {
+		order = get_order(size * MAX_OBJS_PER_PAGE) - 1;
+		goto out;
+	}
+
 	/*
 	 * Attempt to find best configuration for a slab. This
 	 * works by first attempting to generate a layout with
@@ -3865,14 +3867,8 @@ static inline int calculate_order(unsigned int size)
 	 * We were unable to place multiple objects in a slab. Now
 	 * lets see if we can place a single object there.
 	 */
-	order = calc_slab_order(size, 1, slub_max_order, 1);
-	if (order <= slub_max_order)
-		return order;
-
-	/*
-	 * Doh this slab cannot be placed using slub_max_order.
-	 */
-	order = calc_slab_order(size, 1, MAX_ORDER, 1);
+	order = max_t(unsigned int, slub_min_order, get_order(size));
+out:
 	if (order < MAX_ORDER)
 		return order;
 	return -ENOSYS;
Signature · 1 line
2.30.2
#1 lore raw
Quoted 32 lines · Wonhyuk Yang

To calculate order, calc_slab_order() is called repeatly changing the fract_leftover. Thus, the branch which is not dependent on fract_leftover is executed repeatly. So make it run only once.

Plus, when min_object reached to 1, we set fract_leftover to 1. In this case, we can calculate order by max(slub_min_order, get_order(size)) instead of calling calc_slab_order().

No functional impact expected.

---

 mm/slub.c | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)
mm/slub.c +0 -3
index ed5c2c03a47a..1fe4d62b72b8 100644
@@ -3795,9 +3795,6 @@ static inline unsigned int calc_slab_order(unsigned int size,
 	unsigned int min_order = slub_min_order;
 	unsigned int order;
 
-	if (order_objects(min_order, size) > MAX_OBJS_PER_PAGE)
-		return get_order(size * MAX_OBJS_PER_PAGE) - 1;
-
 	for (order = max(min_order, (unsigned int)get_order(min_objects * size));
 			order <= max_order; order++) {
 

mm/slub.c static inline int calculate_order(unsigned int size) · line 3823

Quoted 7 lines
mm/slub.c +4 -0
@@ -3820,6 +3817,11 @@ static inline int calculate_order(unsigned int size)
 	unsigned int max_objects;
 	unsigned int nr_cpus;
 
+	if (unlikely(order_objects(slub_min_order, size) > MAX_OBJS_PER_PAGE)) {
+		order = get_order(size * MAX_OBJS_PER_PAGE) - 1;
+		goto out;
+	}

Hm interestingly, both before and after your patch, MAX_OBJS_PER_PAGE might be theoretically overflowed not by slub_min_order, but then with higher orders. Seems to be prevented only as a side-effect of fragmentation close to none, thus higher orders not attempted. Would be maybe less confusing to check that explicitly. Even if that's wasteful, but this is not really perf critical code.

Quoted 7 lines
mm/slub.c +1 -0
+
 	/*
 	 * Attempt to find best configuration for a slab. This
 	 * works by first attempting to generate a layout with
@@ -3865,14 +3867,8 @@ static inline int calculate_order(unsigned int size)
 	 * We were unable to place multiple objects in a slab. Now
 	 * lets see if we can place a single object there.

mm/slub.c static inline int calculate_order(unsigned int size) · line 3870

Quoted 10 lines
mm/slub.c +1 -8
 	 */
-	order = calc_slab_order(size, 1, slub_max_order, 1);
-	if (order <= slub_max_order)
-		return order;
-
-	/*
-	 * Doh this slab cannot be placed using slub_max_order.
-	 */
-	order = calc_slab_order(size, 1, MAX_ORDER, 1);
+	order = max_t(unsigned int, slub_min_order, get_order(size));

If we failed to assign order above, then AFAICS it means even slub_min_order will not give us more than 1 object per slub. Thus it doesn't make sense to use it in a max() formula, and we can just se get_order(), no?

Quoted 4 lines
mm/slub.c +1 -0
+out:
 	if (order < MAX_ORDER)
 		return order;
 	return -ENOSYS;
#2 lore raw
Quoted 37 lines · Vlastimil Babka
Quoted 35 lines · Wonhyuk Yang

To calculate order, calc_slab_order() is called repeatly changing the fract_leftover. Thus, the branch which is not dependent on fract_leftover is executed repeatly. So make it run only once.

Plus, when min_object reached to 1, we set fract_leftover to 1. In this case, we can calculate order by max(slub_min_order, get_order(size)) instead of calling calc_slab_order().

No functional impact expected.

---

 mm/slub.c | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)
mm/slub.c +0 -3
index ed5c2c03a47a..1fe4d62b72b8 100644
@@ -3795,9 +3795,6 @@ static inline unsigned int calc_slab_order(unsigned int size,
      unsigned int min_order = slub_min_order;
      unsigned int order;
 
-     if (order_objects(min_order, size) > MAX_OBJS_PER_PAGE)
-             return get_order(size * MAX_OBJS_PER_PAGE) - 1;
-
      for (order = max(min_order, (unsigned int)get_order(min_objects * size));
                      order <= max_order; order++) {
 
@@ -3820,6 +3817,11 @@ static inline int calculate_order(unsigned int size)
      unsigned int max_objects;
      unsigned int nr_cpus;
 
Quoted 11 lines

mm/slub.c static inline int calculate_order(unsigned int size) · line 3823

Quoted 4 lines
mm/slub.c +4 -0
+     if (unlikely(order_objects(slub_min_order, size) > MAX_OBJS_PER_PAGE)) {
+             order = get_order(size * MAX_OBJS_PER_PAGE) - 1;
+             goto out;
+     }

Hm interestingly, both before and after your patch, MAX_OBJS_PER_PAGE might be theoretically overflowed not by slub_min_order, but then with higher orders. Seems to be prevented only as a side-effect of fragmentation close to none, thus higher orders not attempted. Would be maybe less confusing to check that explicitly. Even if that's wasteful, but this is not really perf critical code.

Yes, I agree that checking the overflow of object number explicitly is better even if
it is almost impossible. But it checked repeatedly by calling calc_slab_order(). It
seems to me that is unnecessary doesn't it?

Quoted 14 lines
Quoted 14 lines
mm/slub.c +1 -5
+
      /*
       * Attempt to find best configuration for a slab. This
       * works by first attempting to generate a layout with
@@ -3865,14 +3867,8 @@ static inline int calculate_order(unsigned int size)
       * We were unable to place multiple objects in a slab. Now
       * lets see if we can place a single object there.
       */
-     order = calc_slab_order(size, 1, slub_max_order, 1);
-     if (order <= slub_max_order)
-             return order;
-
-     /*
Quoted 8 lines

mm/slub.c static inline int calculate_order(unsigned int size) · line 3870

Quoted 4 lines
mm/slub.c +1 -3
-      * Doh this slab cannot be placed using slub_max_order.
-      */
-     order = calc_slab_order(size, 1, MAX_ORDER, 1);
+     order = max_t(unsigned int, slub_min_order, get_order(size));

If we failed to assign order above, then AFAICS it means even slub_min_order will not give us more than 1 object per slub. Thus it doesn't make sense to use it in a max() formula, and we can just se get_order(), no?

That's sounds reasonable. When it reached here, we don't need to keep the slub_min_order.

Quoted 6 lines
Quoted 5 lines
mm/slub.c +1 -0
+out:
      if (order < MAX_ORDER)
              return order;
      return -ENOSYS;
#3 lore raw
Quoted 12 lines · Wonhyuk Yang
Quoted 11 lines

mm/slub.c static inline int calculate_order(unsigned int size) · line 3823

Quoted 4 lines
mm/slub.c +4 -0
+     if (unlikely(order_objects(slub_min_order, size) > MAX_OBJS_PER_PAGE)) {
+             order = get_order(size * MAX_OBJS_PER_PAGE) - 1;
+             goto out;
+     }

Hm interestingly, both before and after your patch, MAX_OBJS_PER_PAGE might be theoretically overflowed not by slub_min_order, but then with higher orders. Seems to be prevented only as a side-effect of fragmentation close to none, thus higher orders not attempted. Would be maybe less confusing to check that explicitly. Even if that's wasteful, but this is not really perf critical code.

Quoted 5 lines

Yes, I agree that checking the overflow of object number explicitly is better even if
it is almost impossible. But it checked repeatedly by calling calc_slab_order(). It
seems to me that is unnecessary doesn't it?

Yeah I'm OK with the goal of your patch.