Skip to contentSkip to conversation

sched/fair: Complete EEVDF

RFC PATCH In review Peter Zijlstra 10 participants 77 messages Apr 5, 2024 – Jul 2, 2024 1 message not in the archive Scheduler

lore
Share & tools
lore

Peter Zijlstra

author maintainer
#0 lore raw

cover sched/fair: Complete EEVDF

Hi all,

I'm slowly crawling back out of the hole and trying to get back to work. Availability is still low on my end, but I'll try and respond to some email.

Anyway, in order to get my feet wet again with sitting behind a computer, find here a few patches that should functionally complete the EEVDF journey.

This very much includes the new interface that exposes the extra parameter that EEVDF has. I've chosen to use sched_attr::sched_runtime for this over a nice-like value because some workloads actually know their slice length (can be dynamically measured in the same way as for deadline using CLOCK_THREAD_CPUTIME_ID) and using the real request size is much more effective than some relative measure.

[[ using too short a request size will increase job preemption overhead,
using too long a request size will decrease timeliness ]]

The whole delayed-dequeue thing is I think a fundamental thing that was missing from the EEVDF paper. Without something like this EEVDF will simply not work right. IIRC this was mentioned to me many years ago when people worked on BFQ iosched and ran into this same issue. Time had erased the critical aspect of this note and I had to re-discover it again.

Also, I think Ben expressed concern that preserving lag over long periods doesn't make sense a while back.

The implementation presented here is one that should work with our cgroup mess and keeps most of the ugly inside fair.c unlike previous versions which puked all over the core scheduler code.

Critically cfs-cgroup throttling is not tested, and cgroups are only tested in so far that a systemd infected machine now boots (took a bit).

Other than that, it works well enough to build the next kernel and it passes the few trivial latency-slice tests I ran.

Anyway, please have a poke and let me know...

author maintainer
#1 lore raw

RFC PATCH 1/10 sched/eevdf: Add feature comments

---
 kernel/sched/features.h |    7 +++++++
 1 file changed, 7 insertions(+)
kernel/sched/features.h +7 -0
@@ -5,7 +5,14 @@
  * sleep+wake cycles. EEVDF placement strategy #1, #2 if disabled.
  */
 SCHED_FEAT(PLACE_LAG, true)
+/*
+ * Give new tasks half a slice to ease into the competition.
+ */
 SCHED_FEAT(PLACE_DEADLINE_INITIAL, true)
+/*
+ * Inhibit (wakeup) preemption until the current task has either matched the
+ * 0-lag point or until is has exhausted it's slice.
+ */
 SCHED_FEAT(RUN_TO_PARITY, true)
 
 /*
author maintainer
#2 lore raw

RFC PATCH 2/10 sched/eevdf: Remove min_vruntime_copy

In reply to Peter Zijlstra

Since commit e8f331bcc270 ("sched/smp: Use lag to simplify cross-runqueue placement") the min_vruntime_copy is no longer used.

---
 kernel/sched/fair.c  |    5 ++---
 kernel/sched/sched.h |    4 ----
 2 files changed, 2 insertions(+), 7 deletions(-)
kernel/sched/fair.c +2 -3
@@ -780,8 +780,7 @@ static void update_min_vruntime(struct c
 	}
 
 	/* ensure we never gain time by being placed backwards. */
-	u64_u32_store(cfs_rq->min_vruntime,
-		      __update_min_vruntime(cfs_rq, vruntime));
+	cfs_rq->min_vruntime = __update_min_vruntime(cfs_rq, vruntime);
 }
 
 static inline bool __entity_less(struct rb_node *a, const struct rb_node *b)
@@ -12876,7 +12875,7 @@ static void set_next_task_fair(struct rq
 void init_cfs_rq(struct cfs_rq *cfs_rq)
 {
 	cfs_rq->tasks_timeline = RB_ROOT_CACHED;
-	u64_u32_store(cfs_rq->min_vruntime, (u64)(-(1LL << 20)));
+	cfs_rq->min_vruntime = (u64)(-(1LL << 20));
 #ifdef CONFIG_SMP
 	raw_spin_lock_init(&cfs_rq->removed.lock);
 #endif
kernel/sched/sched.h +0 -4
@@ -599,10 +599,6 @@ struct cfs_rq {
 	u64			min_vruntime_fi;
 #endif
 
-#ifndef CONFIG_64BIT
-	u64			min_vruntime_copy;
-#endif
-
 	struct rb_root_cached	tasks_timeline;
 
 	/*
author maintainer
#3 lore raw

RFC PATCH 3/10 sched/fair: Cleanup pick_task_fair() vs throttle

In reply to Peter Zijlstra

Per 54d27365cae8 ("sched/fair: Prevent throttling in early pick_next_task_fair()") the reason check_cfs_rq_runtime() is under the 'if (curr)' check is to ensure the (downward) traversal does not result in an empty cfs_rq.

But then the pick_task_fair() 'copy' of all this made it restart the traversal anyway, so that seems to solve the issue too.

---
 kernel/sched/fair.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/sched/fair.c +3 -3
@@ -8435,11 +8435,11 @@ static struct task_struct *pick_task_fai
 				update_curr(cfs_rq);
 			else
 				curr = NULL;
-
-			if (unlikely(check_cfs_rq_runtime(cfs_rq)))
-				goto again;
 		}
 
+		if (unlikely(check_cfs_rq_runtime(cfs_rq)))
+			goto again;
+
 		se = pick_next_entity(cfs_rq);
 		cfs_rq = group_cfs_rq(se);
 	} while (cfs_rq);
reviewer Reviewed-by on v1 3/10
#4 lore raw

Re: [RFC][PATCH 03/10] sched/fair: Cleanup pick_task_fair() vs throttle

Quoted 7 lines · Peter Zijlstra

Per 54d27365cae8 ("sched/fair: Prevent throttling in early pick_next_task_fair()") the reason check_cfs_rq_runtime() is under the 'if (curr)' check is to ensure the (downward) traversal does not result in an empty cfs_rq.

But then the pick_task_fair() 'copy' of all this made it restart the traversal anyway, so that seems to solve the issue too.

Yeah, putting the check_cfs_rq_runtime inside of that condition was specific to the exact pnt_fair code, and the specific places that nr_running was and was not checked. pick_task_fair doesn't care about any of that, and if instead put_prev manages to throttle the picked task, we can still successfully switch to it for a moment.

Quoted 23 lines
---
 kernel/sched/fair.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/sched/fair.c +3 -3
@@ -8435,11 +8435,11 @@ static struct task_struct *pick_task_fai
 				update_curr(cfs_rq);
 			else
 				curr = NULL;
-
-			if (unlikely(check_cfs_rq_runtime(cfs_rq)))
-				goto again;
 		}
 
+		if (unlikely(check_cfs_rq_runtime(cfs_rq)))
+			goto again;
+
 		se = pick_next_entity(cfs_rq);
 		cfs_rq = group_cfs_rq(se);
 	} while (cfs_rq);
author maintainer
#5 lore raw

RFC PATCH 4/10 sched/fair: Cleanup pick_task_fair()s curr

In reply to Peter Zijlstra

With 4c456c9ad334 ("sched/fair: Remove unused 'curr' argument from pick_next_entity()") curr is no longer being used, so no point in clearing it.

---
 kernel/sched/fair.c |   10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)
kernel/sched/fair.c +2 -8
@@ -8427,15 +8427,9 @@ static struct task_struct *pick_task_fai
 		return NULL;
 
 	do {
-		struct sched_entity *curr = cfs_rq->curr;
-
 		/* When we pick for a remote RQ, we'll not have done put_prev_entity() */
-		if (curr) {
-			if (curr->on_rq)
-				update_curr(cfs_rq);
-			else
-				curr = NULL;
-		}
+		if (cfs_rq->curr && cfs_rq->curr->on_rq)
+			update_curr(cfs_rq);
 
 		if (unlikely(check_cfs_rq_runtime(cfs_rq)))
 			goto again;
author maintainer
#6 lore raw

RFC PATCH 5/10 sched/fair: Unify pick_{,next_}_task_fair()

In reply to Peter Zijlstra

Implement pick_next_task_fair() in terms of pick_task_fair() to de-duplicate the pick loop.

More importantly, this makes all the pick loops use the state-invariant form, which is useful to introduce further re-try conditions in later patches.

---
 kernel/sched/fair.c |   60 ++++++----------------------------------------------
 1 file changed, 8 insertions(+), 52 deletions(-)
kernel/sched/fair.c +8 -52
@@ -8415,7 +8415,6 @@ static void check_preempt_wakeup_fair(st
 	resched_curr(rq);
 }
 
-#ifdef CONFIG_SMP
 static struct task_struct *pick_task_fair(struct rq *rq)
 {
 	struct sched_entity *se;
@@ -8427,7 +8426,7 @@ static struct task_struct *pick_task_fai
 		return NULL;
 
 	do {
-		/* When we pick for a remote RQ, we'll not have done put_prev_entity() */
+		/* Might not have done put_prev_entity() */
 		if (cfs_rq->curr && cfs_rq->curr->on_rq)
 			update_curr(cfs_rq);
 
@@ -8440,19 +8439,20 @@ static struct task_struct *pick_task_fai
 
 	return task_of(se);
 }
-#endif
 
 struct task_struct *
 pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
 {
-	struct cfs_rq *cfs_rq = &rq->cfs;
 	struct sched_entity *se;
 	struct task_struct *p;
+	struct cfs_rq *cfs_rq;
 	int new_tasks;
 
 again:
-	if (!sched_fair_runnable(rq))
+	p = pick_task_fair(rq);
+	if (!p)
 		goto idle;
+	se = &p->se;
 
 #ifdef CONFIG_FAIR_GROUP_SCHED
 	if (!prev || prev->sched_class != &fair_sched_class)
@@ -8464,46 +8464,7 @@ pick_next_task_fair(struct rq *rq, struc
 	 *
 	 * Therefore attempt to avoid putting and setting the entire cgroup
 	 * hierarchy, only change the part that actually changes.
-	 */
-
-	do {
-		struct sched_entity *curr = cfs_rq->curr;
-
-		/*
-		 * Since we got here without doing put_prev_entity() we also
-		 * have to consider cfs_rq->curr. If it is still a runnable
-		 * entity, update_curr() will update its vruntime, otherwise
-		 * forget we've ever seen it.
-		 */
-		if (curr) {
-			if (curr->on_rq)
-				update_curr(cfs_rq);
-			else
-				curr = NULL;
-
-			/*
-			 * This call to check_cfs_rq_runtime() will do the
-			 * throttle and dequeue its entity in the parent(s).
-			 * Therefore the nr_running test will indeed
-			 * be correct.
-			 */
-			if (unlikely(check_cfs_rq_runtime(cfs_rq))) {
-				cfs_rq = &rq->cfs;
-
-				if (!cfs_rq->nr_running)
-					goto idle;
-
-				goto simple;
-			}
-		}
-
-		se = pick_next_entity(cfs_rq);
-		cfs_rq = group_cfs_rq(se);
-	} while (cfs_rq);
-
-	p = task_of(se);
-
-	/*
+	 *
 	 * Since we haven't yet done put_prev_entity and if the selected task
 	 * is a different task than we started out with, try and touch the
 	 * least amount of cfs_rqs.
@@ -8535,13 +8496,8 @@ pick_next_task_fair(struct rq *rq, struc
 	if (prev)
 		put_prev_task(rq, prev);
 
-	do {
-		se = pick_next_entity(cfs_rq);
-		set_next_entity(cfs_rq, se);
-		cfs_rq = group_cfs_rq(se);
-	} while (cfs_rq);
-
-	p = task_of(se);
+	for_each_sched_entity(se)
+		set_next_entity(cfs_rq_of(se), se);
 
 done: __maybe_unused;
 #ifdef CONFIG_SMP
#7 lore raw

Re: [RFC][PATCH 05/10] sched/fair: Unify pick_{,next_}_task_fair()

Greetings (good to see you back).

Quoted 16 lines · Peter Zijlstra
(quoted diff) +0 -0
@@ -8440,19 +8439,20 @@ static struct task_struct *pick_task_fai


        return task_of(se);
 }
-#endif

 struct task_struct *
 pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) {
-       struct cfs_rq *cfs_rq = &rq->cfs;
        struct sched_entity *se;
        struct task_struct *p;
+       struct cfs_rq *cfs_rq;
        int new_tasks;

 again:

Quoted 8 lines

-       if (!sched_fair_runnable(rq))
+       p = pick_task_fair(rq);
+       if (!p)
                goto idle;
+       se = &p->se;

 #ifdef CONFIG_FAIR_GROUP_SCHED
        if (!prev || prev->sched_class != &fair_sched_class)

Those who dodge GROUP_SCHED overhead receive a shiny new unused variable warning.

-Mike

author maintainer
#8 lore raw

RFC PATCH 6/10 sched: Allow sched_class::dequeue_task() to fail

In reply to Peter Zijlstra

Change the function signature of sched_class::dequeue_task() to return a boolean, allowing future patches to 'fail' dequeue.

---
 kernel/sched/core.c      |    7 +++++--
 kernel/sched/deadline.c  |    4 +++-
 kernel/sched/fair.c      |    4 +++-
 kernel/sched/idle.c      |    3 ++-
 kernel/sched/rt.c        |    4 +++-
 kernel/sched/sched.h     |    2 +-
 kernel/sched/stop_task.c |    3 ++-
 7 files changed, 19 insertions(+), 8 deletions(-)
kernel/sched/core.c +5 -2
@@ -2119,7 +2119,10 @@ static inline void enqueue_task(struct r
 		sched_core_enqueue(rq, p);
 }
 
-static inline void dequeue_task(struct rq *rq, struct task_struct *p, int flags)
+/*
+ * Must only return false when DEQUEUE_SLEEP.
+ */
+static inline bool dequeue_task(struct rq *rq, struct task_struct *p, int flags)
 {
 	if (sched_core_enabled(rq))
 		sched_core_dequeue(rq, p, flags);
@@ -2133,7 +2136,7 @@ static inline void dequeue_task(struct r
 	}
 
 	uclamp_rq_dec(rq, p);
-	p->sched_class->dequeue_task(rq, p, flags);
+	return p->sched_class->dequeue_task(rq, p, flags);
 }
 
 void activate_task(struct rq *rq, struct task_struct *p, int flags)
kernel/sched/deadline.c +3 -1
@@ -1842,7 +1842,7 @@ static void enqueue_task_dl(struct rq *r
 		enqueue_pushable_dl_task(rq, p);
 }
 
-static void dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags)
+static bool dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags)
 {
 	update_curr_dl(rq);
 
@@ -1852,6 +1852,8 @@ static void dequeue_task_dl(struct rq *r
 	dequeue_dl_entity(&p->dl, flags);
 	if (!p->dl.dl_throttled && !dl_server(&p->dl))
 		dequeue_pushable_dl_task(rq, p);
+
+	return true;
 }
 
 /*
kernel/sched/fair.c +3 -1
@@ -6828,7 +6828,7 @@ static void set_next_buddy(struct sched_
  * decreased. We remove the task from the rbtree and
  * update the fair scheduling stats:
  */
-static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
+static bool dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 {
 	struct cfs_rq *cfs_rq;
 	struct sched_entity *se = &p->se;
@@ -6896,6 +6896,8 @@ static void dequeue_task_fair(struct rq
 dequeue_throttle:
 	util_est_update(&rq->cfs, p, task_sleep);
 	hrtick_update(rq);
+
+	return true;
 }
 
 #ifdef CONFIG_SMP
kernel/sched/idle.c +2 -1
@@ -486,13 +486,14 @@ struct task_struct *pick_next_task_idle(
  * It is not legal to sleep in the idle task - print a warning
  * message if some code attempts to do it:
  */
-static void
+static bool
 dequeue_task_idle(struct rq *rq, struct task_struct *p, int flags)
 {
 	raw_spin_rq_unlock_irq(rq);
 	printk(KERN_ERR "bad: scheduling from the idle thread!\n");
 	dump_stack();
 	raw_spin_rq_lock_irq(rq);
+	return true;
 }
 
 /*
kernel/sched/rt.c +3 -1
@@ -1493,7 +1493,7 @@ enqueue_task_rt(struct rq *rq, struct ta
 		enqueue_pushable_task(rq, p);
 }
 
-static void dequeue_task_rt(struct rq *rq, struct task_struct *p, int flags)
+static bool dequeue_task_rt(struct rq *rq, struct task_struct *p, int flags)
 {
 	struct sched_rt_entity *rt_se = &p->rt;
 
@@ -1501,6 +1501,8 @@ static void dequeue_task_rt(struct rq *r
 	dequeue_rt_entity(rt_se, flags);
 
 	dequeue_pushable_task(rq, p);
+
+	return true;
 }
 
 /*
kernel/sched/sched.h +1 -1
@@ -2278,7 +2278,7 @@ struct sched_class {
 #endif
 
 	void (*enqueue_task) (struct rq *rq, struct task_struct *p, int flags);
-	void (*dequeue_task) (struct rq *rq, struct task_struct *p, int flags);
+	bool (*dequeue_task) (struct rq *rq, struct task_struct *p, int flags);
 	void (*yield_task)   (struct rq *rq);
 	bool (*yield_to_task)(struct rq *rq, struct task_struct *p);
 
kernel/sched/stop_task.c +2 -1
@@ -57,10 +57,11 @@ enqueue_task_stop(struct rq *rq, struct
 	add_nr_running(rq, 1);
 }
 
-static void
+static bool
 dequeue_task_stop(struct rq *rq, struct task_struct *p, int flags)
 {
 	sub_nr_running(rq, 1);
+	return true;
 }
 
 static void yield_task_stop(struct rq *rq)
author maintainer
#9 lore raw

RFC PATCH 7/10 sched/fair: Re-organize dequeue_task_fair()

In reply to Peter Zijlstra

Working towards delaying dequeue, notably also inside the hierachy, rework dequeue_task_fair() such that it can 'resume' an interrupted hierarchy walk.

---
 kernel/sched/fair.c |   82 ++++++++++++++++++++++++++++++++++------------------
 1 file changed, 55 insertions(+), 27 deletions(-)
kernel/sched/fair.c +55 -27
@@ -6824,33 +6824,45 @@ enqueue_task_fair(struct rq *rq, struct
 static void set_next_buddy(struct sched_entity *se);
 
 /*
- * The dequeue_task method is called before nr_running is
- * decreased. We remove the task from the rbtree and
- * update the fair scheduling stats:
+ * Basically dequeue_task_fair(), except it can deal with dequeue_entity()
+ * failing half-way through and resume the dequeue later.
+ *
+ * Returns:
+ * -1 - dequeue delayed
+ *  0 - dequeue throttled
+ *  1 - dequeue complete
  */
-static bool dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
+static int dequeue_entities(struct rq *rq, struct sched_entity *se, int flags)
 {
-	struct cfs_rq *cfs_rq;
-	struct sched_entity *se = &p->se;
-	int task_sleep = flags & DEQUEUE_SLEEP;
-	int idle_h_nr_running = task_has_idle_policy(p);
 	bool was_sched_idle = sched_idle_rq(rq);
+	bool task_sleep = flags & DEQUEUE_SLEEP;
+	struct task_struct *p = NULL;
+	struct cfs_rq *cfs_rq;
+	int idle_h_nr_running;
 
-	util_est_dequeue(&rq->cfs, p);
+	if (entity_is_task(se)) {
+		p = task_of(se);
+		idle_h_nr_running = task_has_idle_policy(p);
+	} else {
+		idle_h_nr_running = cfs_rq_is_idle(group_cfs_rq(se));
+	}
 
 	for_each_sched_entity(se) {
 		cfs_rq = cfs_rq_of(se);
 		dequeue_entity(cfs_rq, se, flags);
 
-		cfs_rq->h_nr_running--;
-		cfs_rq->idle_h_nr_running -= idle_h_nr_running;
+		/* h_nr_running is the hierachical count of tasks */
+		if (p) {
+			cfs_rq->h_nr_running--;
+			cfs_rq->idle_h_nr_running -= idle_h_nr_running;
 
-		if (cfs_rq_is_idle(cfs_rq))
-			idle_h_nr_running = 1;
+			if (cfs_rq_is_idle(cfs_rq))
+				idle_h_nr_running = 1;
+		}
 
 		/* end evaluation on encountering a throttled cfs_rq */
 		if (cfs_rq_throttled(cfs_rq))
-			goto dequeue_throttle;
+			return 0;
 
 		/* Don't dequeue parent if it has other entities besides us */
 		if (cfs_rq->load.weight) {
@@ -6870,33 +6882,49 @@ static bool dequeue_task_fair(struct rq
 	for_each_sched_entity(se) {
 		cfs_rq = cfs_rq_of(se);
 
+		// XXX avoid these load updates for delayed dequeues ?
 		update_load_avg(cfs_rq, se, UPDATE_TG);
 		se_update_runnable(se);
 		update_cfs_group(se);
 
-		cfs_rq->h_nr_running--;
-		cfs_rq->idle_h_nr_running -= idle_h_nr_running;
+		if (p) {
+			cfs_rq->h_nr_running--;
+			cfs_rq->idle_h_nr_running -= idle_h_nr_running;
 
-		if (cfs_rq_is_idle(cfs_rq))
-			idle_h_nr_running = 1;
+			if (cfs_rq_is_idle(cfs_rq))
+				idle_h_nr_running = 1;
+		}
 
 		/* end evaluation on encountering a throttled cfs_rq */
 		if (cfs_rq_throttled(cfs_rq))
-			goto dequeue_throttle;
+			return 0;
+	}
+
+	if (p) {
+		sub_nr_running(rq, 1);
 
+		/* balance early to pull high priority tasks */
+		if (unlikely(!was_sched_idle && sched_idle_rq(rq)))
+			rq->next_balance = jiffies;
 	}
 
-	/* At this point se is NULL and we are at root level*/
-	sub_nr_running(rq, 1);
+	return 1;
+}
 
-	/* balance early to pull high priority tasks */
-	if (unlikely(!was_sched_idle && sched_idle_rq(rq)))
-		rq->next_balance = jiffies;
+/*
+ * The dequeue_task method is called before nr_running is
+ * decreased. We remove the task from the rbtree and
+ * update the fair scheduling stats:
+ */
+static bool dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
+{
+	util_est_dequeue(&rq->cfs, p);
 
-dequeue_throttle:
-	util_est_update(&rq->cfs, p, task_sleep);
-	hrtick_update(rq);
+	if (dequeue_entities(rq, &p->se, flags) < 0)
+		return false;
 
+	util_est_update(&rq->cfs, p, flags & DEQUEUE_SLEEP);
+	hrtick_update(rq);
 	return true;
 }
 
author maintainer
#10 lore raw

RFC PATCH 8/10 sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra

Extend / fix 86bfbb7ce4f6 ("sched/fair: Add lag based placement") by noting that lag is fundamentally a temporal measure. It should not be carried around indefinitely.

OTOH it should also not be instantly discarded, doing so will allow a task to game the system by purposefully (micro) sleeping at the end of its time quantum.

Since lag is intimately tied to the virtual time base, a wall-time based decay is also insufficient, notably competition is required for any of this to make sense.

Instead, delay the dequeue and keep the 'tasks' on the runqueue, competing until they are eligible.

Strictly speaking, we only care about keeping them until the 0-lag point, but that is a difficult proposition, instead carry them around until they get picked again, and dequeue them at that point.

Since we should have dequeued them at the 0-lag point, truncate lag (eg. don't let them earn positive lag).

XXX test the cfs-throttle stuff

---
 include/linux/sched.h   |    1 
 kernel/sched/core.c     |   22 +++++--
 kernel/sched/fair.c     |  148 +++++++++++++++++++++++++++++++++++++++++++-----
 kernel/sched/features.h |   12 +++
 kernel/sched/sched.h    |    2 
 5 files changed, 167 insertions(+), 18 deletions(-)
include/linux/sched.h +1 -0
@@ -542,6 +542,7 @@ struct sched_entity {
 
 	struct list_head		group_node;
 	unsigned int			on_rq;
+	unsigned int			sched_delayed;
 
 	u64				exec_start;
 	u64				sum_exec_runtime;
kernel/sched/core.c +18 -4
@@ -2154,10 +2154,14 @@ void activate_task(struct rq *rq, struct
 
 void deactivate_task(struct rq *rq, struct task_struct *p, int flags)
 {
-	WRITE_ONCE(p->on_rq, (flags & DEQUEUE_SLEEP) ? 0 : TASK_ON_RQ_MIGRATING);
-	ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+	bool sleep = flags & DEQUEUE_SLEEP;
 
-	dequeue_task(rq, p, flags);
+	if (dequeue_task(rq, p, flags)) {
+		WRITE_ONCE(p->on_rq, sleep ? 0 : TASK_ON_RQ_MIGRATING);
+		ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+	} else {
+		SCHED_WARN_ON(!sleep); /* only sleep can fail */
+	}
 }
 
 static inline int __normal_prio(int policy, int rt_prio, int nice)
@@ -3858,12 +3862,17 @@ static int ttwu_runnable(struct task_str
 
 	rq = __task_rq_lock(p, &rf);
 	if (task_on_rq_queued(p)) {
+		update_rq_clock(rq);
+		if (p->se.sched_delayed) {
+			/* mustn't run a delayed task */
+			SCHED_WARN_ON(task_on_cpu(rq, p));
+			enqueue_task(rq, p, ENQUEUE_DELAYED);
+		}
 		if (!task_on_cpu(rq, p)) {
 			/*
 			 * When on_rq && !on_cpu the task is preempted, see if
 			 * it should preempt the task that is current now.
 			 */
-			update_rq_clock(rq);
 			wakeup_preempt(rq, p, wake_flags);
 		}
 		ttwu_do_wakeup(p);
@@ -4243,11 +4252,16 @@ int try_to_wake_up(struct task_struct *p
 		 * case the whole 'p->on_rq && ttwu_runnable()' case below
 		 * without taking any locks.
 		 *
+		 * Specifically, given current runs ttwu() we must be before
+		 * schedule()'s deactivate_task(), as such this must not
+		 * observe sched_delayed.
+		 *
 		 * In particular:
 		 *  - we rely on Program-Order guarantees for all the ordering,
 		 *  - we're serialized against set_special_state() by virtue of
 		 *    it disabling IRQs (this allows not taking ->pi_lock).
 		 */
+		SCHED_WARN_ON(p->se.sched_delayed);
 		if (!ttwu_state_match(p, state, &success))
 			goto out;
 
kernel/sched/fair.c +134 -14

280 lines not on this page · raw message

kernel/sched/features.h +12 -0
@@ -29,6 +29,18 @@ SCHED_FEAT(NEXT_BUDDY, false)
 SCHED_FEAT(CACHE_HOT_BUDDY, true)
 
 /*
+ * Delay dequeueing tasks until they get selected or woken.
+ *
+ * By delaying the dequeue for non-eligible tasks, they remain in the
+ * competition and can burn off their negative lag. When they get selected
+ * they'll have positive lag by definition.
+ *
+ * DELAY_ZERO clips the lag on dequeue (or wakeup) to 0.
+ */
+SCHED_FEAT(DELAY_DEQUEUE, true)
+SCHED_FEAT(DELAY_ZERO, true)
+
+/*
  * Allow wakeup-time preemption of the current task:
  */
 SCHED_FEAT(WAKEUP_PREEMPTION, true)
kernel/sched/sched.h +2 -0
@@ -2245,6 +2245,7 @@ extern const u32		sched_prio_to_wmult[40
 #define DEQUEUE_MOVE		0x04 /* Matches ENQUEUE_MOVE */
 #define DEQUEUE_NOCLOCK		0x08 /* Matches ENQUEUE_NOCLOCK */
 #define DEQUEUE_MIGRATING	0x100 /* Matches ENQUEUE_MIGRATING */
+#define DEQUEUE_DELAYED		0x200 /* Matches ENQUEUE_DELAYED */
 
 #define ENQUEUE_WAKEUP		0x01
 #define ENQUEUE_RESTORE		0x02
@@ -2260,6 +2261,7 @@ extern const u32		sched_prio_to_wmult[40
 #endif
 #define ENQUEUE_INITIAL		0x80
 #define ENQUEUE_MIGRATING	0x100
+#define ENQUEUE_DELAYED		0x200
 
 #define RETRY_TASK		((void *)-1UL)
 
#11 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 19 lines · Peter Zijlstra

Extend / fix 86bfbb7ce4f6 ("sched/fair: Add lag based placement") by noting that lag is fundamentally a temporal measure. It should not be carried around indefinitely.

OTOH it should also not be instantly discarded, doing so will allow a task to game the system by purposefully (micro) sleeping at the end of its time quantum.

Since lag is intimately tied to the virtual time base, a wall-time based decay is also insufficient, notably competition is required for any of this to make sense.

Instead, delay the dequeue and keep the 'tasks' on the runqueue, competing until they are eligible.

Strictly speaking, we only care about keeping them until the 0-lag point, but that is a difficult proposition, instead carry them around until they get picked again, and dequeue them at that point.

Quoted 7 lines

Since we should have dequeued them at the 0-lag point, truncate lag (eg. don't let them earn positive lag).

XXX test the cfs-throttle stuff

---

Tested schbench on xeon server, which has 240 CPUs/2 sockets. schbench -m 2 -r 100
the result seems ok to me.

baseline:
NO_DELAY_DEQUEUE
NO_DELAY_ZERO
Wakeup Latencies percentiles (usec) runtime 100 (s) (1658446 total samples)

	  50.0th: 5          (361126 samples)
	  90.0th: 11         (654121 samples)
	* 99.0th: 25         (123032 samples)
	  99.9th: 673        (13845 samples)
	  min=1, max=8337
Request Latencies percentiles (usec) runtime 100 (s) (1662381 total samples)
	  50.0th: 14992      (524771 samples)
	  90.0th: 15344      (657370 samples)
	* 99.0th: 15568      (129769 samples)
	  99.9th: 15888      (10017 samples)
	  min=3529, max=43841
RPS percentiles (requests) runtime 100 (s) (101 total samples)
	  20.0th: 16544      (37 samples)
	* 50.0th: 16608      (30 samples)
	  90.0th: 16736      (31 samples)
	  min=16403, max=17698

average rps: 16623.81

DELAY_DEQUEUE
DELAY_ZERO
Wakeup Latencies percentiles (usec) runtime 100 (s) (1668161 total samples)

	  50.0th: 6          (394867 samples)
	  90.0th: 12         (653021 samples)
	* 99.0th: 31         (142636 samples)
	  99.9th: 755        (14547 samples)
	  min=1, max=5226
Request Latencies percentiles (usec) runtime 100 (s) (1671859 total samples)
	  50.0th: 14384      (511809 samples)
	  90.0th: 14992      (653508 samples)
	* 99.0th: 15408      (149257 samples)
	  99.9th: 15984      (12090 samples)
	  min=3546, max=38360
RPS percentiles (requests) runtime 100 (s) (101 total samples)
	  20.0th: 16672      (45 samples)
	* 50.0th: 16736      (52 samples)
	  90.0th: 16736      (0 samples)
	  min=16629, max=16800

average rps: 16718.59

The 99th wakeup latency increases a little bit, and should be in the acceptible range(25 -> 31 us). Meanwhile the throughput increases accordingly. Here are the possible reason I can think of:

1. wakeup latency: The time to find an eligible entity in the tree during wakeup might take longer - if there are more delayed-dequeue tasks in the tree.
2. throughput: Inhibit task dequeue can decrease the ratio to touch the task group's load_avg: dequeue_entity()-> { update_load_avg(), update_cfs_group()), which reduces the cache contention among CPUs, and improves throughput.

kernel/sched/fair.c static void clear_buddies(struct cfs_rq · line 5379

Quoted 8 lines
kernel/sched/fair.c +8 -0
+	} else {
+		bool sleep = flags & DEQUEUE_SLEEP;
+
+		SCHED_WARN_ON(sleep && se->sched_delayed);
+		update_curr(cfs_rq);
+
+		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !entity_eligible(cfs_rq, se)) {

Regarding the elibigle check, it was found that there could be an overflow issue, and it brings false negative of entity_eligible(), which was described here:
https://lore.kernel.org/lkml/[email protected]/
and also reported on another machine
https://lore.kernel.org/lkml/[email protected]/
I don't have good idea to avoid that overflow properly, while I'm trying to reproduce it locally, do you have any guidance on how to address it?

thanks,
Chenyu

author maintainer
#12 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 2 lines · Chen Yu

The 99th wakeup latency increases a little bit, and should be in the acceptible range(25 -> 31 us).

Ah, my test runs haven't been stable enough to observe that.

Quoted 6 lines

Meanwhile the throughput increases accordingly. Here are the possible reason I can think of:

1. wakeup latency: The time to find an eligible entity in the tree during wakeup might take longer - if there are more delayed-dequeue tasks in the tree.

Another possible cause might be that previously a schedule() would be 1 dequeue, 1 pick.

But now it can be much more variable, a pick can basically do N dequeues and N+1 picks.

So not only do we do more picks, but if you're focussed on worst case latency, it goes up, because we can do multiple dequeues for a single pick.

If we find this to really be a problem, I had some half baked ideas to fix it, but it added significant complexity, so keep it simple until need proves we need more etc.

Quoted 3 lines

2. throughput: Inhibit task dequeue can decrease the ratio to touch the task group's load_avg: dequeue_entity()-> { update_load_avg(), update_cfs_group()), which reduces the cache contention among CPUs, and improves throughput.

Ah, yes, there's that.

Quoted 9 lines

kernel/sched/fair.c static void clear_buddies(struct cfs_rq · line 5379

Quoted 8 lines
kernel/sched/fair.c +8 -0
+	} else {
+		bool sleep = flags & DEQUEUE_SLEEP;
+
+		SCHED_WARN_ON(sleep && se->sched_delayed);
+		update_curr(cfs_rq);
+
+		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !entity_eligible(cfs_rq, se)) {
Quoted 7 lines

Regarding the elibigle check, it was found that there could be an overflow issue, and it brings false negative of entity_eligible(), which was described here:
https://lore.kernel.org/lkml/[email protected]/
and also reported on another machine
https://lore.kernel.org/lkml/[email protected]/
I don't have good idea to avoid that overflow properly, while I'm trying to reproduce it locally, do you have any guidance on how to address it?

I have not yet seen those, let me go stare at them now. Thanks!

#13 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 19 lines
Quoted 3 lines

The 99th wakeup latency increases a little bit, and should be in the acceptible range(25 -> 31 us).

Ah, my test runs haven't been stable enough to observe that.

Quoted 6 lines

Meanwhile the throughput increases accordingly. Here are the possible reason I can think of:

1. wakeup latency: The time to find an eligible entity in the tree during wakeup might take longer - if there are more delayed-dequeue tasks in the tree.

Another possible cause might be that previously a schedule() would be 1 dequeue, 1 pick.

But now it can be much more variable, a pick can basically do N dequeues and N+1 picks.

Quoted 7 lines

So not only do we do more picks, but if you're focussed on worst case latency, it goes up, because we can do multiple dequeues for a single pick.

If we find this to really be a problem, I had some half baked ideas to fix it, but it added significant complexity, so keep it simple until need proves we need more etc.

I have an alternative approach to delayed-dequeue inspired by the original CFS implementation.

The idea is to keep the task's vruntime when it goes to sleep.
When the task is woken up, see if the lag is positive at the woken time, if it is the case, clamp it to 0 by setting vruntime to avg_vruntime().

<Sleep>

In dequeue_entity(): Remove the task from runqueue, but keep the task's vruntime, and do not calculate vlag at this time.

<Wake Up on the same CPU>

In enqueue_entity():
1. Do not call place_entity().
2. If the task's vruntume is less than the cfs_rq's avg_vruntime(),
set the task's vruntime to avg_vruntime(), and update the task's deadline according to its timeslice.
3. Insert the task into the runqueue.

<Wake Up on different CPU>

In migrate_task_rq_fair():
1. Calculate the task's vlag as if it is on the original cfs_rq.
2. Set the task's vlag to 0 if it is positive.

In enqueue_entity(): Use place_entity() to calculate the task's new vruntime and deadline according to the vlag and the new runqueue before inserting it into the runqueue.

author maintainer
#14 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 6 lines · Yan-Jie Wang

I have an alternative approach to delayed-dequeue inspired by the original CFS implementation.

The idea is to keep the task's vruntime when it goes to sleep. When the task is woken up, see if the lag is positive at the woken time, if it is the case, clamp it to 0 by setting vruntime to avg_vruntime().

Problem is that avg_vruntime() can go backwards, eg by dequeueing a negative lag task. This gets really hard to argue about real quick.

Keeping the task competing (delaying the dequeue) is by far the simplest solution -- conceptually.

The code just became a total mess because of cgroups :/

#15 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra

Hello Peter,

Quoted 15 lines · Peter Zijlstra

Extend / fix 86bfbb7ce4f6 ("sched/fair: Add lag based placement") by noting that lag is fundamentally a temporal measure. It should not be carried around indefinitely.

OTOH it should also not be instantly discarded, doing so will allow a task to game the system by purposefully (micro) sleeping at the end of its time quantum.

Since lag is intimately tied to the virtual time base, a wall-time based decay is also insufficient, notably competition is required for any of this to make sense.

Instead, delay the dequeue and keep the 'tasks' on the runqueue, competing until they are eligible.

Quoted 8 lines

Strictly speaking, we only care about keeping them until the 0-lag point, but that is a difficult proposition, instead carry them around until they get picked again, and dequeue them at that point.

Since we should have dequeued them at the 0-lag point, truncate lag (eg. don't let them earn positive lag).

XXX test the cfs-throttle stuff

I ran into a few issues when testing the series on top of tip:sched/core at commit 4475cd8bfd9b ("sched/balancing: Simplify the sg_status bitmask and use separate ->overloaded and ->overutilized flags"). All of these splats surfaced when running unixbench with Delayed Dequeue (echoing NO_DELAY_DEQUEUE to /sys/kernel/debug/sched/features seems to make the system stable when running Unixbench spawn)

Unixbench (https://github.com/kdlucas/byte-unixbench.git) command:

	./Run spawn -c 512

Splats appear soon into the run. Following are the splats and their corresponding code blocks from my 3rd Generation EPYC system (2 x 64C/128T):

1. NULL pointer dereferencing in can_migrate_task():

	BUG: kernel NULL pointer dereference, address: 0000000000000040
	#PF: supervisor read access in kernel mode
	#PF: error_code(0x0000) - not-present page
	PGD 0 P4D 0
	Oops: 0000 [#1] PREEMPT SMP NOPTI
	CPU: 154 PID: 1507736 Comm: spawn Not tainted 6.9.0-rc1-test+ #958
	Hardware name: Dell Inc. PowerEdge R6525/024PW1, BIOS 2.7.3 03/30/2022
	RIP: 0010:can_migrate_task+0x2b/0x6c0
	Code: ...
	RSP: 0018:ffffb6bb9e6a3bc0 EFLAGS: 00010086
	RAX: 0000000000000000 RBX: ffffb6bb9e6a3c80 RCX: ffff90ad0d209400
	RDX: 0000000000000008 RSI: ffffb6bb9e6a3c80 RDI: ffff90eb3b236438
	RBP: ffff90eb3b236438 R08: 0000005c743512ab R09: ffffffffffff0000
	R10: 0000000000000001 R11: 0000000000000100 R12: ffff90eb3b236438
	R13: ffff90eb3b2364f0 R14: ffff90eb3b6359c0 R15: ffff90eb3b6359c0
	FS:  0000000000000000(0000) GS:ffff90eb3df00000(0000) knlGS:0000000000000000
	CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
	CR2: 0000000000000040 CR3: 000000807da3c006 CR4: 0000000000770ef0
	PKRU: 55555554
	Call Trace:
	<TASK>
	? __die+0x24/0x70
	? page_fault_oops+0x14a/0x510
	? exc_page_fault+0x77/0x170
	? asm_exc_page_fault+0x26/0x30
	? can_migrate_task+0x2b/0x6c0
	sched_balance_rq+0x7a8/0x1190
	sched_balance_newidle+0x1e2/0x490
	pick_next_task_fair+0x36/0x4a0
	__schedule+0x1c0/0x1710
	? srso_alias_return_thunk+0x5/0xfbef5
	? refill_stock+0x1a/0x30
	? srso_alias_return_thunk+0x5/0xfbef5
	? obj_cgroup_uncharge_pages+0x4d/0xd0
	do_task_dead+0x42/0x50
	do_exit+0x777/0xad0
	do_group_exit+0x30/0x80
	__x64_sys_exit_group+0x18/0x20
	do_syscall_64+0x79/0x120
	? srso_alias_return_thunk+0x5/0xfbef5
	? srso_alias_return_thunk+0x5/0xfbef5
	? irqentry_exit_to_user_mode+0x5b/0x170
	entry_SYSCALL_64_after_hwframe+0x6c/0x74
	RIP: 0033:0x7f963a6eac31
	Code: Unable to access opcode bytes at 0x7f963a6eac07.
	RSP: 002b:00007ffc6b7158c8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
	RAX: ffffffffffffffda RBX: 00007f963a816a00 RCX: 00007f963a6eac31
	RDX: 000000000000003c RSI: 00000000000000e7 RDI: 0000000000000000
	RBP: 0000000000000000 R08: ffffffffffffff80 R09: 0000000000000020
	R10: 0000000000000000 R11: 0000000000000246 R12: 00007f963a816a00
	R13: 0000000000000000 R14: 00007f963a81bee8 R15: 00007f963a81bf00
	</TASK>
	Modules linked in: ...
	CR2: 0000000000000040
	---[ end trace 0000000000000000 ]---
    $ scripts/faddr2line vmlinux can_migrate_task+0x2b/0x6c0
    can_migrate_task+0x2b/0x6c0:
    throttled_lb_pair at kernel/sched/fair.c:5738
    (inlined by) can_migrate_task at kernel/sched/fair.c:9090
    Corresponds to:

	static inline int throttled_lb_pair(struct task_group *tg,
                                    int src_cpu, int dest_cpu)
	{
	        struct cfs_rq *src_cfs_rq, *dest_cfs_rq;

	        src_cfs_rq = tg->cfs_rq[src_cpu];  /* <----- Here -----< */
	        dest_cfs_rq = tg->cfs_rq[dest_cpu];

	        return throttled_hierarchy(src_cfs_rq) ||
	               throttled_hierarchy(dest_cfs_rq);
	}

	(inlined by)
	int can_migrate_task(struct task_struct *p, struct lb_env *env)
	{
		/* Called here */
		if (throttled_lb_pair(task_group(p), env->src_cpu, env->dst_cpu))
			return 0;

		...
	}

2. A NULL pointer dereferencing in pick_next_task_fair():

	BUG: kernel NULL pointer dereference, address: 0000000000000098
	#PF: supervisor read access in kernel mode
	#PF: error_code(0x0000) - not-present page
	PGD 0 P4D 0
	Oops: 0000 [#1] PREEMPT SMP NOPTI
	CPU: 107 PID: 1206665 Comm: spawn Tainted: G        W          6.9.0-rc1-test+ #958
	Hardware name: Dell Inc. PowerEdge R6525/024PW1, BIOS 2.7.3 03/30/2022
	RIP: 0010:pick_next_task_fair+0x327/0x4a0
	Code: ...
	RSP: 0018:ffffb613c212fd28 EFLAGS: 00010002
	RAX: 0000004ed2799383 RBX: 0000000000000000 RCX: ffff8f65baf3f800
	RDX: ffff8f65baf3ca00 RSI: 0000000000000000 RDI: 000000825ae302ab
	RBP: ffff8f64b13b59c0 R08: 0000000000000015 R09: 0000000000000314
	R10: 0000000000000001 R11: 0000000000000001 R12: ffff8f261ac199c0
	R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
	FS:  00007f768b79e740(0000) GS:ffff8f64b1380000(0000) knlGS:0000000000000000
	CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
	CR2: 0000000000000098 CR3: 00000040d1010006 CR4: 0000000000770ef0
	PKRU: 55555554
	Call Trace:
	 <TASK>
	 ? __die+0x24/0x70
	 ? page_fault_oops+0x14a/0x510
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 ? report_bug+0x18e/0x1a0
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 ? exc_page_fault+0x77/0x170
	 ? asm_exc_page_fault+0x26/0x30
	 ? pick_next_task_fair+0x327/0x4a0
	 ? pick_next_task_fair+0x320/0x4a0
	 __schedule+0x1c0/0x1710
	 ? release_task+0x2fc/0x4c0
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 schedule+0x30/0x120
	 syscall_exit_to_user_mode+0x98/0x1b0
	 do_syscall_64+0x85/0x120
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 ? __count_memcg_events+0x69/0x100
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 ? count_memcg_events.constprop.0+0x1a/0x30
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 ? handle_mm_fault+0x17d/0x2e0
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 ? do_user_addr_fault+0x33d/0x6f0
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 ? srso_alias_return_thunk+0x5/0xfbef5
	 ? irqentry_exit_to_user_mode+0x5b/0x170
	 entry_SYSCALL_64_after_hwframe+0x6c/0x74
	RIP: 0033:0x7f768b4eab57
	Code: ...
	RSP: 002b:00007fff5f6e2018 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
	RAX: 000000000026d13b RBX: 00007f768b7ee040 RCX: 00007f768b4eab57
	RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
	RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
	R10: 00007f768b79ea10 R11: 0000000000000246 R12: 0000000000000001
	R13: 000055b811d1a140 R14: 000055b811d1cd88 R15: 00007f768b7ee040
	 </TASK>
	Modules linked in: ...
	CR2: 0000000000000098
	---[ end trace 0000000000000000 ]---
    $ scripts/faddr2line vmlinux pick_next_task_fair+0x327/0x4a0
    pick_next_task_fair+0x327/0x4a0:
    is_same_group at kernel/sched/fair.c:418
    (inlined by) pick_next_task_fair at kernel/sched/fair.c:8625
	struct cfs_rq *
	is_same_group(struct sched_entity *se, struct sched_entity *pse)
	{
		if (se->cfs_rq == pse->cfs_rq) /* <----- HERE -----< */
			return se->cfs_rq;

		return NULL;
	}

	(inlined by)
	struct task_struct *
	pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
	{
		...
		if (prev != p) {
			...
			while (!(cfs_rq = is_same_group(se, pse) /* <---- HERE ----< */)) { 
				...
			}
		...
		}
		...
	}

3. A NULL Pointer dereferencing in __dequeue_entity():

	BUG: kernel NULL pointer dereference, address: 0000000000000000
	#PF: supervisor read access in kernel mode
	#PF: error_code(0x0000) - not-present page
	PGD 0 P4D 0
	Oops: 0000 [#1] PREEMPT SMP NOPTI
	CPU: 95 PID: 60896 Comm: spawn Not tainted 6.9.0-rc1-test+ #958
	Hardware name: Dell Inc. PowerEdge R6525/024PW1, BIOS 2.7.3 03/30/2022
	RIP: 0010:__rb_erase_color+0x88/0x260
	Code: ...
	RSP: 0018:ffffab158755fc08 EFLAGS: 00010046
	RAX: 0000000000000000 RBX: ffffffff841314b0 RCX: 0000000017fc8dd0
	RDX: 0000000000000000 RSI: ffff8decfb1fe450 RDI: ffff8decf80bcdd0
	RBP: ffff8decf80bcdd0 R08: ffff8decf80bcdd0 R09: ffffffffffffbb60
	R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
	R13: ffff8decfb1fe450 R14: ffff8ded0ec03400 R15: ffff8decfb1fe400
	FS:  00007f1ded0a2740(0000) GS:ffff8e2bb0d80000(0000) knlGS:0000000000000000
	CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
	CR2: 0000000000000000 CR3: 00000040e66f0005 CR4: 0000000000770ef0
	PKRU: 55555554
	Call Trace:
	<TASK>
	? __die+0x24/0x70
	? page_fault_oops+0x14a/0x510
	? exc_page_fault+0x77/0x170
	? asm_exc_page_fault+0x26/0x30
	? __pfx_min_vruntime_cb_rotate+0x10/0x10
	? __rb_erase_color+0x88/0x260
	__dequeue_entity+0x1b7/0x310
	set_next_entity+0xc0/0x1e0
	pick_next_task_fair+0x355/0x4a0
	__schedule+0x1c0/0x1710
	? native_queued_spin_lock_slowpath+0x2a4/0x2f0
	schedule+0x30/0x120
	do_wait+0xad/0x100
	kernel_wait4+0xa9/0x150
	? __pfx_child_wait_callback+0x10/0x10
	do_syscall_64+0x79/0x120
	? srso_alias_return_thunk+0x5/0xfbef5
	? __count_memcg_events+0x69/0x100
	? srso_alias_return_thunk+0x5/0xfbef5
	? count_memcg_events.constprop.0+0x1a/0x30
	? srso_alias_return_thunk+0x5/0xfbef5
	? handle_mm_fault+0x17d/0x2e0
	? srso_alias_return_thunk+0x5/0xfbef5
	? do_user_addr_fault+0x33d/0x6f0
	? srso_alias_return_thunk+0x5/0xfbef5
	? srso_alias_return_thunk+0x5/0xfbef5
	? irqentry_exit_to_user_mode+0x5b/0x170
	entry_SYSCALL_64_after_hwframe+0x6c/0x74
	RIP: 0033:0x7f1deceea3ea
	Code: ...
	RSP: 002b:00007ffd7fd37ca8 EFLAGS: 00000246 ORIG_RAX: 000000000000003d
	RAX: ffffffffffffffda RBX: 00007ffd7fd37cb4 RCX: 00007f1deceea3ea
	RDX: 0000000000000000 RSI: 00007ffd7fd37cb4 RDI: 00000000ffffffff
	RBP: 0000000000000002 R08: 00000000000136f5 R09: 0000000000000000
	R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffd7fd37dd8
	R13: 000055fd8debf140 R14: 000055fd8dec1d88 R15: 00007f1ded0f2040
	</TASK>
	Modules linked in: ...
	CR2: 0000000000000000
	---[ end trace 0000000000000000 ]---

Note: I only ran into this issue with unixbench spawn. A bunch of other benchmarks (hackbench, stream, tbench, netperf, schbench, other variants of unixbench) ran fine without bringing down the machine.

Attaching my config below in case this in config specific.

Quoted 10 lines
---
 include/linux/sched.h   |    1 
 kernel/sched/core.c     |   22 +++++--
 kernel/sched/fair.c     |  148 +++++++++++++++++++++++++++++++++++++++++++-----
 kernel/sched/features.h |   12 +++
 kernel/sched/sched.h    |    2 
 5 files changed, 167 insertions(+), 18 deletions(-)
Quoted 2 lines

[..snip..]

--
Thanks and Regards,
Prateek

#
# Automatically generated file; DO NOT EDIT.
# Linux/x86 6.9.0-rc1 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0" CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=110400
CONFIG_CLANG_VERSION=0
CONFIG_AS_IS_GNU=y
CONFIG_AS_VERSION=23800
CONFIG_LD_IS_BFD=y
CONFIG_LD_VERSION=23800
CONFIG_LLD_VERSION=0
CONFIG_CC_CAN_LINK=y
CONFIG_CC_CAN_LINK_STATIC=y CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y CONFIG_CC_HAS_ASM_GOTO_TIED_OUTPUT=y CONFIG_GCC_ASM_GOTO_OUTPUT_WORKAROUND=y CONFIG_TOOLS_SUPPORT_RELR=y
CONFIG_CC_HAS_ASM_INLINE=y CONFIG_CC_HAS_NO_PROFILE_FN_ATTR=y CONFIG_PAHOLE_VERSION=125
CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_TABLE_SORT=y CONFIG_THREAD_INFO_IN_TASK=y

#
# General setup
#
CONFIG_INIT_ENV_ARG_LIMIT=32
# CONFIG_COMPILE_TEST is not set
# CONFIG_WERROR is not set
# CONFIG_UAPI_HEADER_TEST is not set
CONFIG_LOCALVERSION="-test"
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_BUILD_SALT=""
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_HAVE_KERNEL_LZ4=y
CONFIG_HAVE_KERNEL_ZSTD=y
# CONFIG_KERNEL_GZIP is not set
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_KERNEL_LZ4 is not set
CONFIG_KERNEL_ZSTD=y
CONFIG_DEFAULT_INIT=""
CONFIG_DEFAULT_HOSTNAME="(none)"
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_SYSVIPC_COMPAT=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_WATCH_QUEUE=y
CONFIG_CROSS_MEMORY_ATTACH=y
CONFIG_USELIB=y
CONFIG_AUDIT=y
CONFIG_HAVE_ARCH_AUDITSYSCALL=y
CONFIG_AUDITSYSCALL=y

#
# IRQ subsystem
#
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_GENERIC_IRQ_MIGRATION=y
CONFIG_HARDIRQS_SW_RESEND=y
CONFIG_GENERIC_IRQ_CHIP=y
CONFIG_IRQ_DOMAIN=y
CONFIG_IRQ_DOMAIN_HIERARCHY=y
CONFIG_GENERIC_MSI_IRQ=y
CONFIG_IRQ_MSI_IOMMU=y
CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR=y
CONFIG_GENERIC_IRQ_RESERVATION_MODE=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
# CONFIG_GENERIC_IRQ_DEBUGFS is not set

# end of IRQ subsystem

CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_ARCH_CLOCKSOURCE_INIT=y
CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST_IDLE=y
CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_HAVE_POSIX_CPU_TIMERS_TASK_WORK=y
CONFIG_POSIX_CPU_TIMERS_TASK_WORK=y
CONFIG_CONTEXT_TRACKING=y
CONFIG_CONTEXT_TRACKING_IDLE=y

#
# Timers subsystem
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
CONFIG_NO_HZ_IDLE=y
# CONFIG_NO_HZ_FULL is not set
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_CLOCKSOURCE_WATCHDOG_MAX_SKEW_US=125

# end of Timers subsystem

CONFIG_BPF=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y

#
# BPF subsystem
#
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_JIT=y
CONFIG_BPF_JIT_ALWAYS_ON=y
CONFIG_BPF_JIT_DEFAULT_ON=y
CONFIG_BPF_UNPRIV_DEFAULT_OFF=y
# CONFIG_BPF_PRELOAD is not set
CONFIG_BPF_LSM=y

# end of BPF subsystem

CONFIG_PREEMPT_BUILD=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
CONFIG_PREEMPT_COUNT=y
CONFIG_PREEMPTION=y
CONFIG_PREEMPT_DYNAMIC=y
CONFIG_SCHED_CORE=y

#
# CPU/Task time and stats accounting
#
CONFIG_TICK_CPU_ACCOUNTING=y
# CONFIG_VIRT_CPU_ACCOUNTING_GEN is not set
# CONFIG_IRQ_TIME_ACCOUNTING is not set
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_BSD_PROCESS_ACCT_V3=y
CONFIG_TASKSTATS=y
CONFIG_TASK_DELAY_ACCT=y
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_PSI=y
# CONFIG_PSI_DEFAULT_DISABLED is not set

# end of CPU/Task time and stats accounting

CONFIG_CPU_ISOLATION=y

#
# RCU Subsystem
#
CONFIG_TREE_RCU=y
CONFIG_PREEMPT_RCU=y
# CONFIG_RCU_EXPERT is not set
CONFIG_TREE_SRCU=y
CONFIG_TASKS_RCU_GENERIC=y
CONFIG_TASKS_RCU=y
CONFIG_TASKS_RUDE_RCU=y
CONFIG_TASKS_TRACE_RCU=y
CONFIG_RCU_STALL_COMMON=y
CONFIG_RCU_NEED_SEGCBLIST=y

# end of RCU Subsystem

# CONFIG_IKCONFIG is not set
CONFIG_IKHEADERS=m
CONFIG_LOG_BUF_SHIFT=18
CONFIG_LOG_CPU_MAX_BUF_SHIFT=12
# CONFIG_PRINTK_INDEX is not set
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y

#
# Scheduler features
#
CONFIG_UCLAMP_TASK=y
CONFIG_UCLAMP_BUCKETS_COUNT=5

# end of Scheduler features

CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH=y CONFIG_CC_HAS_INT128=y CONFIG_CC_IMPLICIT_FALLTHROUGH="-Wimplicit-fallthrough=5" CONFIG_GCC10_NO_ARRAY_BOUNDS=y CONFIG_CC_NO_ARRAY_BOUNDS=y CONFIG_GCC_NO_STRINGOP_OVERFLOW=y CONFIG_CC_NO_STRINGOP_OVERFLOW=y CONFIG_ARCH_SUPPORTS_INT128=y CONFIG_NUMA_BALANCING=y CONFIG_NUMA_BALANCING_DEFAULT_ENABLED=y CONFIG_CGROUPS=y
CONFIG_PAGE_COUNTER=y
# CONFIG_CGROUP_FAVOR_DYNMODS is not set CONFIG_MEMCG=y
CONFIG_MEMCG_KMEM=y
CONFIG_BLK_CGROUP=y
CONFIG_CGROUP_WRITEBACK=y
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_CFS_BANDWIDTH=y
# CONFIG_RT_GROUP_SCHED is not set CONFIG_SCHED_MM_CID=y
CONFIG_UCLAMP_TASK_GROUP=y
CONFIG_CGROUP_PIDS=y
CONFIG_CGROUP_RDMA=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_HUGETLB=y
CONFIG_CPUSETS=y
CONFIG_PROC_PID_CPUSET=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_BPF=y
CONFIG_CGROUP_MISC=y
# CONFIG_CGROUP_DEBUG is not set CONFIG_SOCK_CGROUP_DATA=y
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_TIME_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
CONFIG_CHECKPOINT_RESTORE=y CONFIG_SCHED_AUTOGROUP=y
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_RD_LZ4=y
CONFIG_RD_ZSTD=y
CONFIG_BOOT_CONFIG=y
# CONFIG_BOOT_CONFIG_FORCE is not set
# CONFIG_BOOT_CONFIG_EMBED is not set CONFIG_INITRAMFS_PRESERVE_MTIME=y CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set CONFIG_LD_ORPHAN_WARN=y CONFIG_LD_ORPHAN_WARN_LEVEL="warn" CONFIG_SYSCTL=y
CONFIG_HAVE_UID16=y CONFIG_SYSCTL_EXCEPTION_TRACE=y CONFIG_HAVE_PCSPKR_PLATFORM=y
CONFIG_EXPERT=y
CONFIG_UID16=y
CONFIG_MULTIUSER=y
CONFIG_SGETMASK_SYSCALL=y
CONFIG_SYSFS_SYSCALL=y
CONFIG_FHANDLE=y
CONFIG_POSIX_TIMERS=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_FUTEX_PI=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_IO_URING=y
CONFIG_ADVISE_SYSCALLS=y
CONFIG_MEMBARRIER=y
CONFIG_KCMP=y
CONFIG_RSEQ=y
# CONFIG_DEBUG_RSEQ is not set CONFIG_CACHESTAT_SYSCALL=y
CONFIG_PC104=y
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_SELFTEST is not set CONFIG_KALLSYMS_ALL=y CONFIG_KALLSYMS_ABSOLUTE_PERCPU=y CONFIG_KALLSYMS_BASE_RELATIVE=y CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE=y CONFIG_HAVE_PERF_EVENTS=y CONFIG_GUEST_PERF_EVENTS=y

#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set

# end of Kernel Performance Events And Counters

CONFIG_SYSTEM_DATA_VERIFICATION=y
CONFIG_PROFILING=y
CONFIG_TRACEPOINTS=y

#
# Kexec and crash features
#
CONFIG_CRASH_RESERVE=y
CONFIG_VMCORE_INFO=y
CONFIG_KEXEC_CORE=y
CONFIG_HAVE_IMA_KEXEC=y
CONFIG_KEXEC=y
CONFIG_KEXEC_FILE=y
CONFIG_KEXEC_SIG=y
# CONFIG_KEXEC_SIG_FORCE is not set
CONFIG_KEXEC_BZIMAGE_VERIFY_SIG=y
CONFIG_KEXEC_JUMP=y
CONFIG_CRASH_DUMP=y
CONFIG_CRASH_HOTPLUG=y
CONFIG_CRASH_MAX_MEMORY_RANGES=8192

# end of Kexec and crash features
# end of General setup

CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_MMU=y
CONFIG_ARCH_MMAP_RND_BITS_MIN=28
CONFIG_ARCH_MMAP_RND_BITS_MAX=32
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MIN=8
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MAX=16
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_AUDIT_ARCH=y
CONFIG_HAVE_INTEL_TXT=y
CONFIG_X86_64_SMP=y
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_DYNAMIC_PHYSICAL_MASK=y
CONFIG_PGTABLE_LEVELS=5
CONFIG_CC_HAS_SANE_STACKPROTECTOR=y

#
# Processor type and features
#
CONFIG_SMP=y
CONFIG_X86_X2APIC=y
CONFIG_X86_MPPARSE=y
CONFIG_X86_CPU_RESCTRL=y
# CONFIG_X86_FRED is not set
CONFIG_X86_EXTENDED_PLATFORM=y
CONFIG_X86_NUMACHIP=y
# CONFIG_X86_VSMP is not set
CONFIG_X86_UV=y
# CONFIG_X86_GOLDFISH is not set
# CONFIG_X86_INTEL_MID is not set
CONFIG_X86_INTEL_LPSS=y
CONFIG_X86_AMD_PLATFORM_DEVICE=y
CONFIG_IOSF_MBI=y
CONFIG_IOSF_MBI_DEBUG=y
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
CONFIG_HYPERVISOR_GUEST=y
CONFIG_PARAVIRT=y
CONFIG_PARAVIRT_XXL=y
# CONFIG_PARAVIRT_DEBUG is not set
CONFIG_PARAVIRT_SPINLOCKS=y
CONFIG_X86_HV_CALLBACK_VECTOR=y
CONFIG_XEN=y
CONFIG_XEN_PV=y
CONFIG_XEN_512GB=y
CONFIG_XEN_PV_SMP=y
CONFIG_XEN_PV_DOM0=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_PVHVM_SMP=y
CONFIG_XEN_PVHVM_GUEST=y
CONFIG_XEN_SAVE_RESTORE=y
# CONFIG_XEN_DEBUG_FS is not set
CONFIG_XEN_PVH=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PV_MSR_SAFE=y
CONFIG_KVM_GUEST=y
CONFIG_ARCH_CPUIDLE_HALTPOLL=y
CONFIG_PVH=y
# CONFIG_PARAVIRT_TIME_ACCOUNTING is not set
CONFIG_PARAVIRT_CLOCK=y
CONFIG_JAILHOUSE_GUEST=y
CONFIG_ACRN_GUEST=y
# CONFIG_INTEL_TDX_GUEST is not set
# CONFIG_MK8 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
CONFIG_GENERIC_CPU=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_HAVE_PAE=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_IA32_FEAT_CTL=y
CONFIG_X86_VMX_FEATURE_NAMES=y
CONFIG_PROCESSOR_SELECT=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_HYGON=y
CONFIG_CPU_SUP_CENTAUR=y
CONFIG_CPU_SUP_ZHAOXIN=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
CONFIG_GART_IOMMU=y
CONFIG_BOOT_VESA_SUPPORT=y
CONFIG_MAXSMP=y
CONFIG_NR_CPUS_RANGE_BEGIN=8192
CONFIG_NR_CPUS_RANGE_END=8192
CONFIG_NR_CPUS_DEFAULT=8192
CONFIG_NR_CPUS=8192
CONFIG_SCHED_CLUSTER=y
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
CONFIG_SCHED_MC_PRIO=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y
CONFIG_X86_MCE=y
CONFIG_X86_MCELOG_LEGACY=y
CONFIG_X86_MCE_INTEL=y
CONFIG_X86_MCE_AMD=y
CONFIG_X86_MCE_THRESHOLD=y
CONFIG_X86_MCE_INJECT=m

#
# Performance monitoring
#
CONFIG_PERF_EVENTS_INTEL_UNCORE=y
CONFIG_PERF_EVENTS_INTEL_RAPL=m
CONFIG_PERF_EVENTS_INTEL_CSTATE=m
# CONFIG_PERF_EVENTS_AMD_POWER is not set
CONFIG_PERF_EVENTS_AMD_UNCORE=m
# CONFIG_PERF_EVENTS_AMD_BRS is not set

# end of Performance monitoring

CONFIG_X86_16BIT=y
CONFIG_X86_ESPFIX64=y
CONFIG_X86_VSYSCALL_EMULATION=y
CONFIG_X86_IOPL_IOPERM=y
CONFIG_MICROCODE=y
CONFIG_MICROCODE_LATE_LOADING=y
# CONFIG_MICROCODE_LATE_FORCE_MINREV is not set
CONFIG_X86_MSR=m
CONFIG_X86_CPUID=m
CONFIG_X86_5LEVEL=y
CONFIG_X86_DIRECT_GBPAGES=y
# CONFIG_X86_CPA_STATISTICS is not set
CONFIG_X86_MEM_ENCRYPT=y
CONFIG_AMD_MEM_ENCRYPT=y
CONFIG_NUMA=y
CONFIG_AMD_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
# CONFIG_NUMA_EMU is not set
CONFIG_NODES_SHIFT=10
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_MEMORY_PROBE=y
CONFIG_ARCH_PROC_KCORE_TEXT=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_X86_PMEM_LEGACY_DEVICE=y
CONFIG_X86_PMEM_LEGACY=y
CONFIG_X86_CHECK_BIOS_CORRUPTION=y
CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK=y
CONFIG_MTRR=y
CONFIG_MTRR_SANITIZER=y
CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=1
CONFIG_MTRR_SANITIZER_SPARE_REG_NR_DEFAULT=1
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
CONFIG_X86_UMIP=y
CONFIG_CC_HAS_IBT=y
CONFIG_X86_CET=y
CONFIG_X86_KERNEL_IBT=y
CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS=y
CONFIG_X86_INTEL_TSX_MODE_OFF=y
# CONFIG_X86_INTEL_TSX_MODE_ON is not set
# CONFIG_X86_INTEL_TSX_MODE_AUTO is not set
CONFIG_X86_SGX=y
# CONFIG_X86_USER_SHADOW_STACK is not set
CONFIG_EFI=y
CONFIG_EFI_STUB=y
CONFIG_EFI_HANDOVER_PROTOCOL=y
CONFIG_EFI_MIXED=y
# CONFIG_EFI_FAKE_MEMMAP is not set
CONFIG_EFI_RUNTIME_MAP=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
CONFIG_SCHED_HRTICK=y
CONFIG_ARCH_SUPPORTS_KEXEC=y
CONFIG_ARCH_SUPPORTS_KEXEC_FILE=y
CONFIG_ARCH_SELECTS_KEXEC_FILE=y
CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY=y
CONFIG_ARCH_SUPPORTS_KEXEC_SIG=y
CONFIG_ARCH_SUPPORTS_KEXEC_SIG_FORCE=y
CONFIG_ARCH_SUPPORTS_KEXEC_BZIMAGE_VERIFY_SIG=y
CONFIG_ARCH_SUPPORTS_KEXEC_JUMP=y
CONFIG_ARCH_SUPPORTS_CRASH_DUMP=y
CONFIG_ARCH_SUPPORTS_CRASH_HOTPLUG=y
CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION=y
CONFIG_PHYSICAL_START=0x1000000
CONFIG_RELOCATABLE=y
CONFIG_RANDOMIZE_BASE=y
CONFIG_X86_NEED_RELOCS=y
CONFIG_PHYSICAL_ALIGN=0x200000
CONFIG_DYNAMIC_MEMORY_LAYOUT=y
CONFIG_RANDOMIZE_MEMORY=y
CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING=0xa
# CONFIG_ADDRESS_MASKING is not set
CONFIG_HOTPLUG_CPU=y
# CONFIG_COMPAT_VDSO is not set
CONFIG_LEGACY_VSYSCALL_XONLY=y
# CONFIG_LEGACY_VSYSCALL_NONE is not set
# CONFIG_CMDLINE_BOOL is not set
CONFIG_MODIFY_LDT_SYSCALL=y
# CONFIG_STRICT_SIGALTSTACK_SIZE is not set
CONFIG_HAVE_LIVEPATCH=y
CONFIG_LIVEPATCH=y

# end of Processor type and features

CONFIG_CC_HAS_SLS=y
CONFIG_CC_HAS_RETURN_THUNK=y
CONFIG_CC_HAS_ENTRY_PADDING=y
CONFIG_FUNCTION_PADDING_CFI=11
CONFIG_FUNCTION_PADDING_BYTES=16
CONFIG_CALL_PADDING=y
CONFIG_HAVE_CALL_THUNKS=y
CONFIG_CALL_THUNKS=y
CONFIG_PREFIX_SYMBOLS=y
CONFIG_SPECULATION_MITIGATIONS=y
CONFIG_MITIGATION_PAGE_TABLE_ISOLATION=y
CONFIG_MITIGATION_RETPOLINE=y
CONFIG_MITIGATION_RETHUNK=y
CONFIG_MITIGATION_UNRET_ENTRY=y
CONFIG_MITIGATION_CALL_DEPTH_TRACKING=y
# CONFIG_CALL_THUNKS_DEBUG is not set
CONFIG_MITIGATION_IBPB_ENTRY=y
CONFIG_MITIGATION_IBRS_ENTRY=y
CONFIG_MITIGATION_SRSO=y
# CONFIG_MITIGATION_SLS is not set
# CONFIG_MITIGATION_GDS_FORCE is not set
CONFIG_MITIGATION_RFDS=y
CONFIG_ARCH_HAS_ADD_PAGES=y

#
# Power management and ACPI options
#
CONFIG_ARCH_HIBERNATION_HEADER=y
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
# CONFIG_SUSPEND_SKIP_SYNC is not set
CONFIG_HIBERNATE_CALLBACKS=y
CONFIG_HIBERNATION=y
CONFIG_HIBERNATION_SNAPSHOT_DEV=y
CONFIG_HIBERNATION_COMP_LZO=y
# CONFIG_HIBERNATION_COMP_LZ4 is not set
CONFIG_HIBERNATION_DEF_COMP="lzo"
CONFIG_PM_STD_PARTITION=""
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
# CONFIG_PM_AUTOSLEEP is not set
# CONFIG_PM_USERSPACE_AUTOSLEEP is not set
CONFIG_PM_WAKELOCKS=y
CONFIG_PM_WAKELOCKS_LIMIT=100
CONFIG_PM_WAKELOCKS_GC=y
CONFIG_PM=y
CONFIG_PM_DEBUG=y
CONFIG_PM_ADVANCED_DEBUG=y
# CONFIG_PM_TEST_SUSPEND is not set
CONFIG_PM_SLEEP_DEBUG=y
# CONFIG_DPM_WATCHDOG is not set
CONFIG_PM_TRACE=y
CONFIG_PM_TRACE_RTC=y
CONFIG_PM_CLK=y
CONFIG_PM_GENERIC_DOMAINS=y
CONFIG_WQ_POWER_EFFICIENT_DEFAULT=y
CONFIG_PM_GENERIC_DOMAINS_SLEEP=y
CONFIG_ENERGY_MODEL=y
CONFIG_ARCH_SUPPORTS_ACPI=y
CONFIG_ACPI=y
CONFIG_ACPI_LEGACY_TABLES_LOOKUP=y
CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC=y
CONFIG_ACPI_SYSTEM_POWER_STATES_SUPPORT=y
CONFIG_ACPI_TABLE_LIB=y
CONFIG_ACPI_THERMAL_LIB=y
CONFIG_ACPI_DEBUGGER=y
CONFIG_ACPI_DEBUGGER_USER=y
CONFIG_ACPI_SPCR_TABLE=y
CONFIG_ACPI_FPDT=y
CONFIG_ACPI_LPIT=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_REV_OVERRIDE_POSSIBLE=y
CONFIG_ACPI_EC_DEBUGFS=m
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_FAN=y
CONFIG_ACPI_TAD=m
CONFIG_ACPI_DOCK=y
CONFIG_ACPI_CPU_FREQ_PSS=y
CONFIG_ACPI_PROCESSOR_CSTATE=y
CONFIG_ACPI_PROCESSOR_IDLE=y
CONFIG_ACPI_CPPC_LIB=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_IPMI=m
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_PROCESSOR_AGGREGATOR=m
CONFIG_ACPI_THERMAL=y
CONFIG_ACPI_PLATFORM_PROFILE=m
CONFIG_ACPI_CUSTOM_DSDT_FILE=""
CONFIG_ARCH_HAS_ACPI_TABLE_UPGRADE=y
CONFIG_ACPI_TABLE_UPGRADE=y
CONFIG_ACPI_DEBUG=y
CONFIG_ACPI_PCI_SLOT=y
CONFIG_ACPI_CONTAINER=y
CONFIG_ACPI_HOTPLUG_MEMORY=y
CONFIG_ACPI_HOTPLUG_IOAPIC=y
CONFIG_ACPI_SBS=m
CONFIG_ACPI_HED=y
CONFIG_ACPI_BGRT=y
# CONFIG_ACPI_REDUCED_HARDWARE_ONLY is not set
CONFIG_ACPI_NFIT=m
# CONFIG_NFIT_SECURITY_DEBUG is not set
CONFIG_ACPI_NUMA=y
CONFIG_ACPI_HMAT=y
CONFIG_HAVE_ACPI_APEI=y
CONFIG_HAVE_ACPI_APEI_NMI=y
CONFIG_ACPI_APEI=y
CONFIG_ACPI_APEI_GHES=y
CONFIG_ACPI_APEI_PCIEAER=y
CONFIG_ACPI_APEI_MEMORY_FAILURE=y
CONFIG_ACPI_APEI_EINJ=m
CONFIG_ACPI_APEI_EINJ_CXL=y
# CONFIG_ACPI_APEI_ERST_DEBUG is not set
CONFIG_ACPI_DPTF=y
CONFIG_DPTF_POWER=m
CONFIG_DPTF_PCH_FIVR=m
CONFIG_ACPI_WATCHDOG=y
CONFIG_ACPI_EXTLOG=m
CONFIG_ACPI_ADXL=y
CONFIG_ACPI_CONFIGFS=m
# CONFIG_ACPI_PFRUT is not set
CONFIG_ACPI_PCC=y
# CONFIG_ACPI_FFH is not set
CONFIG_PMIC_OPREGION=y
CONFIG_BYTCRC_PMIC_OPREGION=y
CONFIG_CHTCRC_PMIC_OPREGION=y
CONFIG_XPOWER_PMIC_OPREGION=y
CONFIG_BXT_WC_PMIC_OPREGION=y
CONFIG_CHT_WC_PMIC_OPREGION=y
CONFIG_CHT_DC_TI_PMIC_OPREGION=y
CONFIG_TPS68470_PMIC_OPREGION=y
CONFIG_ACPI_VIOT=y
CONFIG_ACPI_PRMT=y
CONFIG_X86_PM_TIMER=y

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y CONFIG_CPU_FREQ_GOV_ATTR_SET=y CONFIG_CPU_FREQ_GOV_COMMON=y CONFIG_CPU_FREQ_STAT=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set # CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL=y CONFIG_CPU_FREQ_GOV_PERFORMANCE=y CONFIG_CPU_FREQ_GOV_POWERSAVE=y CONFIG_CPU_FREQ_GOV_USERSPACE=y CONFIG_CPU_FREQ_GOV_ONDEMAND=y CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y

#
# CPU frequency scaling drivers
#
CONFIG_X86_INTEL_PSTATE=y
CONFIG_X86_PCC_CPUFREQ=y
CONFIG_X86_AMD_PSTATE=y
CONFIG_X86_AMD_PSTATE_DEFAULT_MODE=3
# CONFIG_X86_AMD_PSTATE_UT is not set
CONFIG_X86_ACPI_CPUFREQ=y
CONFIG_X86_ACPI_CPUFREQ_CPB=y
CONFIG_X86_POWERNOW_K8=y
CONFIG_X86_AMD_FREQ_SENSITIVITY=m
CONFIG_X86_SPEEDSTEP_CENTRINO=y
CONFIG_X86_P4_CLOCKMOD=m

#

# shared options
#
CONFIG_X86_SPEEDSTEP_LIB=m
# end of CPU Frequency scaling

#
# CPU Idle
#
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y
CONFIG_CPU_IDLE_GOV_TEO=y
CONFIG_CPU_IDLE_GOV_HALTPOLL=y
CONFIG_HALTPOLL_CPUIDLE=m

# end of CPU Idle

CONFIG_INTEL_IDLE=y

# end of Power management and ACPI options

#
# Bus options (PCI etc.)
#
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_XEN=y
CONFIG_MMCONF_FAM10H=y
# CONFIG_PCI_CNB20LE_QUIRK is not set
CONFIG_ISA_BUS=y
CONFIG_ISA_DMA_API=y
CONFIG_AMD_NB=y

# end of Bus options (PCI etc.)

#
# Binary Emulations
#
CONFIG_IA32_EMULATION=y
# CONFIG_IA32_EMULATION_DEFAULT_DISABLED is not set # CONFIG_X86_X32_ABI is not set CONFIG_COMPAT_32=y
CONFIG_COMPAT=y CONFIG_COMPAT_FOR_U64_ALIGNMENT=y

# end of Binary Emulations

CONFIG_KVM_COMMON=y
CONFIG_HAVE_KVM_PFNCACHE=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_DIRTY_RING=y
CONFIG_HAVE_KVM_DIRTY_RING_TSO=y
CONFIG_HAVE_KVM_DIRTY_RING_ACQ_REL=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_READONLY_MEM=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_KVM_COMPAT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_HAVE_KVM_NO_POLL=y
CONFIG_KVM_XFER_TO_GUEST_WORK=y
CONFIG_HAVE_KVM_PM_NOTIFIER=y
CONFIG_KVM_GENERIC_HARDWARE_ENABLING=y
CONFIG_KVM_GENERIC_MMU_NOTIFIER=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=m
CONFIG_KVM_WERROR=y
# CONFIG_KVM_SW_PROTECTED_VM is not set
CONFIG_KVM_INTEL=m
CONFIG_X86_SGX_KVM=y
CONFIG_KVM_AMD=m
CONFIG_KVM_AMD_SEV=y
CONFIG_KVM_SMM=y
CONFIG_KVM_HYPERV=y
CONFIG_KVM_XEN=y
# CONFIG_KVM_PROVE_MMU is not set
CONFIG_KVM_EXTERNAL_WRITE_TRACKING=y
CONFIG_KVM_MAX_NR_VCPUS=4096
CONFIG_AS_AVX512=y
CONFIG_AS_SHA1_NI=y
CONFIG_AS_SHA256_NI=y
CONFIG_AS_TPAUSE=y
CONFIG_AS_GFNI=y
CONFIG_AS_WRUSS=y

#
# General architecture-dependent options
#
CONFIG_HOTPLUG_SMT=y
CONFIG_HOTPLUG_CORE_SYNC=y
CONFIG_HOTPLUG_CORE_SYNC_DEAD=y
CONFIG_HOTPLUG_CORE_SYNC_FULL=y
CONFIG_HOTPLUG_SPLIT_STARTUP=y
CONFIG_HOTPLUG_PARALLEL=y
CONFIG_GENERIC_ENTRY=y
CONFIG_KPROBES=y
CONFIG_JUMP_LABEL=y
# CONFIG_STATIC_KEYS_SELFTEST is not set
# CONFIG_STATIC_CALL_SELFTEST is not set
CONFIG_OPTPROBES=y
CONFIG_KPROBES_ON_FTRACE=y
CONFIG_UPROBES=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_ARCH_USE_BUILTIN_BSWAP=y
CONFIG_KRETPROBES=y
CONFIG_KRETPROBE_ON_RETHOOK=y
CONFIG_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_KPROBES_ON_FTRACE=y
CONFIG_ARCH_CORRECT_STACKTRACE_ON_KRETPROBE=y
CONFIG_HAVE_FUNCTION_ERROR_INJECTION=y
CONFIG_HAVE_NMI=y
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_NMI_SUPPORT=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_CONTIGUOUS=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_ARCH_HAS_FORTIFY_SOURCE=y
CONFIG_ARCH_HAS_SET_MEMORY=y
CONFIG_ARCH_HAS_SET_DIRECT_MAP=y
CONFIG_ARCH_HAS_CPU_FINALIZE_INIT=y
CONFIG_ARCH_HAS_CPU_PASID=y
CONFIG_HAVE_ARCH_THREAD_STRUCT_WHITELIST=y
CONFIG_ARCH_WANTS_DYNAMIC_TASK_STRUCT=y
CONFIG_ARCH_WANTS_NO_INSTR=y
CONFIG_HAVE_ASM_MODVERSIONS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_RSEQ=y
CONFIG_HAVE_RUST=y
CONFIG_HAVE_FUNCTION_ARG_ACCESS_API=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_HARDLOCKUP_DETECTOR_PERF=y
CONFIG_HAVE_PERF_REGS=y
CONFIG_HAVE_PERF_USER_STACK_DUMP=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE=y
CONFIG_MMU_GATHER_TABLE_FREE=y
CONFIG_MMU_GATHER_RCU_TABLE_FREE=y
CONFIG_MMU_GATHER_MERGE_VMAS=y
CONFIG_MMU_LAZY_TLB_REFCOUNT=y
CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG=y
CONFIG_ARCH_HAS_NMI_SAFE_THIS_CPU_OPS=y
CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y
CONFIG_HAVE_CMPXCHG_LOCAL=y
CONFIG_HAVE_CMPXCHG_DOUBLE=y
CONFIG_ARCH_WANT_COMPAT_IPC_PARSE_VERSION=y
CONFIG_ARCH_WANT_OLD_COMPAT_IPC=y
CONFIG_HAVE_ARCH_SECCOMP=y
CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
CONFIG_SECCOMP=y
CONFIG_SECCOMP_FILTER=y
# CONFIG_SECCOMP_CACHE_DEBUG is not set
CONFIG_HAVE_ARCH_STACKLEAK=y
CONFIG_HAVE_STACKPROTECTOR=y
CONFIG_STACKPROTECTOR=y
CONFIG_STACKPROTECTOR_STRONG=y
CONFIG_ARCH_SUPPORTS_LTO_CLANG=y
CONFIG_ARCH_SUPPORTS_LTO_CLANG_THIN=y
CONFIG_LTO_NONE=y
CONFIG_ARCH_SUPPORTS_CFI_CLANG=y
CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES=y
CONFIG_HAVE_CONTEXT_TRACKING_USER=y
CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_HAVE_MOVE_PUD=y
CONFIG_HAVE_MOVE_PMD=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD=y
CONFIG_HAVE_ARCH_HUGE_VMAP=y
CONFIG_HAVE_ARCH_HUGE_VMALLOC=y
CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
CONFIG_ARCH_WANT_PMD_MKWRITE=y
CONFIG_HAVE_ARCH_SOFT_DIRTY=y
CONFIG_HAVE_MOD_ARCH_SPECIFIC=y
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK=y
CONFIG_HAVE_SOFTIRQ_ON_OWN_STACK=y
CONFIG_SOFTIRQ_ON_OWN_STACK=y
CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
CONFIG_HAVE_ARCH_MMAP_RND_BITS=y
CONFIG_HAVE_EXIT_THREAD=y
CONFIG_ARCH_MMAP_RND_BITS=28
CONFIG_HAVE_ARCH_MMAP_RND_COMPAT_BITS=y
CONFIG_ARCH_MMAP_RND_COMPAT_BITS=8
CONFIG_HAVE_ARCH_COMPAT_MMAP_BASES=y
CONFIG_HAVE_PAGE_SIZE_4KB=y
CONFIG_PAGE_SIZE_4KB=y
CONFIG_PAGE_SIZE_LESS_THAN_64KB=y
CONFIG_PAGE_SIZE_LESS_THAN_256KB=y
CONFIG_PAGE_SHIFT=12
CONFIG_HAVE_OBJTOOL=y
CONFIG_HAVE_JUMP_LABEL_HACK=y
CONFIG_HAVE_NOINSTR_HACK=y
CONFIG_HAVE_NOINSTR_VALIDATION=y
CONFIG_HAVE_UACCESS_VALIDATION=y
CONFIG_HAVE_STACK_VALIDATION=y
CONFIG_HAVE_RELIABLE_STACKTRACE=y
CONFIG_ISA_BUS_API=y
CONFIG_OLD_SIGSUSPEND3=y
CONFIG_COMPAT_OLD_SIGACTION=y
CONFIG_COMPAT_32BIT_TIME=y
CONFIG_HAVE_ARCH_VMAP_STACK=y
CONFIG_VMAP_STACK=y
CONFIG_HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET=y
CONFIG_RANDOMIZE_KSTACK_OFFSET=y
CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT=y
CONFIG_ARCH_HAS_STRICT_KERNEL_RWX=y
CONFIG_STRICT_KERNEL_RWX=y
CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
CONFIG_STRICT_MODULE_RWX=y
CONFIG_HAVE_ARCH_PREL32_RELOCATIONS=y
CONFIG_ARCH_USE_MEMREMAP_PROT=y
# CONFIG_LOCK_EVENT_COUNTS is not set
CONFIG_ARCH_HAS_MEM_ENCRYPT=y
CONFIG_ARCH_HAS_CC_PLATFORM=y
CONFIG_HAVE_STATIC_CALL=y
CONFIG_HAVE_STATIC_CALL_INLINE=y
CONFIG_HAVE_PREEMPT_DYNAMIC=y
CONFIG_HAVE_PREEMPT_DYNAMIC_CALL=y
CONFIG_ARCH_WANT_LD_ORPHAN_WARN=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_ARCH_SUPPORTS_PAGE_TABLE_CHECK=y
CONFIG_ARCH_HAS_ELFCORE_COMPAT=y
CONFIG_ARCH_HAS_PARANOID_L1D_FLUSH=y
CONFIG_DYNAMIC_SIGFRAME=y
CONFIG_HAVE_ARCH_NODE_DEV_GROUP=y
CONFIG_ARCH_HAS_HW_PTE_YOUNG=y
CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG=y

#
# GCOV-based kernel profiling
#
# CONFIG_GCOV_KERNEL is not set
CONFIG_ARCH_HAS_GCOV_PROFILE_ALL=y

# end of GCOV-based kernel profiling

CONFIG_HAVE_GCC_PLUGINS=y
CONFIG_FUNCTION_ALIGNMENT_4B=y
CONFIG_FUNCTION_ALIGNMENT_16B=y
CONFIG_FUNCTION_ALIGNMENT=16

# end of General architecture-dependent options

CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULES=y
# CONFIG_MODULE_DEBUG is not set
# CONFIG_MODULE_FORCE_LOAD is not set CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
# CONFIG_MODULE_UNLOAD_TAINT_TRACKING is not set CONFIG_MODVERSIONS=y
CONFIG_ASM_MODVERSIONS=y CONFIG_MODULE_SRCVERSION_ALL=y CONFIG_MODULE_SIG=y
# CONFIG_MODULE_SIG_FORCE is not set CONFIG_MODULE_SIG_ALL=y
# CONFIG_MODULE_SIG_SHA256 is not set
# CONFIG_MODULE_SIG_SHA384 is not set CONFIG_MODULE_SIG_SHA512=y
# CONFIG_MODULE_SIG_SHA3_256 is not set
# CONFIG_MODULE_SIG_SHA3_384 is not set
# CONFIG_MODULE_SIG_SHA3_512 is not set CONFIG_MODULE_SIG_HASH="sha512" CONFIG_MODULE_COMPRESS_NONE=y
# CONFIG_MODULE_COMPRESS_GZIP is not set
# CONFIG_MODULE_COMPRESS_XZ is not set
# CONFIG_MODULE_COMPRESS_ZSTD is not set
# CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set CONFIG_MODPROBE_PATH="/sbin/modprobe"
# CONFIG_TRIM_UNUSED_KSYMS is not set CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_BLOCK=y
CONFIG_BLOCK_LEGACY_AUTOLOAD=y CONFIG_BLK_RQ_ALLOC_TIME=y CONFIG_BLK_CGROUP_RWSTAT=y CONFIG_BLK_CGROUP_PUNT_BIO=y CONFIG_BLK_DEV_BSG_COMMON=y
CONFIG_BLK_ICQ=y
CONFIG_BLK_DEV_BSGLIB=y CONFIG_BLK_DEV_INTEGRITY=y CONFIG_BLK_DEV_INTEGRITY_T10=y CONFIG_BLK_DEV_WRITE_MOUNTED=y CONFIG_BLK_DEV_ZONED=y CONFIG_BLK_DEV_THROTTLING=y
# CONFIG_BLK_DEV_THROTTLING_LOW is not set CONFIG_BLK_WBT=y
CONFIG_BLK_WBT_MQ=y
# CONFIG_BLK_CGROUP_IOLATENCY is not set CONFIG_BLK_CGROUP_FC_APPID=y CONFIG_BLK_CGROUP_IOCOST=y CONFIG_BLK_CGROUP_IOPRIO=y
CONFIG_BLK_DEBUG_FS=y
CONFIG_BLK_DEBUG_FS_ZONED=y
CONFIG_BLK_SED_OPAL=y CONFIG_BLK_INLINE_ENCRYPTION=y CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK=y

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
CONFIG_AIX_PARTITION=y
CONFIG_OSF_PARTITION=y
CONFIG_AMIGA_PARTITION=y
CONFIG_ATARI_PARTITION=y
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_LDM_PARTITION=y
# CONFIG_LDM_DEBUG is not set
CONFIG_SGI_PARTITION=y
CONFIG_ULTRIX_PARTITION=y
CONFIG_SUN_PARTITION=y
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
CONFIG_SYSV68_PARTITION=y
CONFIG_CMDLINE_PARTITION=y

# end of Partition Types

CONFIG_BLK_MQ_PCI=y
CONFIG_BLK_MQ_VIRTIO=y
CONFIG_BLK_PM=y
CONFIG_BLOCK_HOLDER_DEPRECATED=y
CONFIG_BLK_MQ_STACKING=y

#
# IO Schedulers
#
CONFIG_MQ_IOSCHED_DEADLINE=y
CONFIG_MQ_IOSCHED_KYBER=m
CONFIG_IOSCHED_BFQ=m
CONFIG_BFQ_GROUP_IOSCHED=y
# CONFIG_BFQ_CGROUP_DEBUG is not set

# end of IO Schedulers

CONFIG_PREEMPT_NOTIFIERS=y
CONFIG_PADATA=y
CONFIG_ASN1=y
CONFIG_UNINLINE_SPIN_UNLOCK=y
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
CONFIG_MUTEX_SPIN_ON_OWNER=y
CONFIG_RWSEM_SPIN_ON_OWNER=y
CONFIG_LOCK_SPIN_ON_OWNER=y
CONFIG_ARCH_USE_QUEUED_SPINLOCKS=y
CONFIG_QUEUED_SPINLOCKS=y
CONFIG_ARCH_USE_QUEUED_RWLOCKS=y
CONFIG_QUEUED_RWLOCKS=y
CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE=y
CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE=y
CONFIG_ARCH_HAS_SYSCALL_WRAPPER=y
CONFIG_FREEZER=y

#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
CONFIG_ELFCORE=y
CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BINFMT_MISC=m
CONFIG_COREDUMP=y

# end of Executable file formats

#
# Memory Management options
#
CONFIG_ZPOOL=y
CONFIG_SWAP=y
CONFIG_ZSWAP=y
# CONFIG_ZSWAP_DEFAULT_ON is not set
# CONFIG_ZSWAP_SHRINKER_DEFAULT_ON is not set # CONFIG_ZSWAP_COMPRESSOR_DEFAULT_DEFLATE is not set CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZO=y
# CONFIG_ZSWAP_COMPRESSOR_DEFAULT_842 is not set # CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZ4 is not set # CONFIG_ZSWAP_COMPRESSOR_DEFAULT_LZ4HC is not set # CONFIG_ZSWAP_COMPRESSOR_DEFAULT_ZSTD is not set CONFIG_ZSWAP_COMPRESSOR_DEFAULT="lzo" CONFIG_ZSWAP_ZPOOL_DEFAULT_ZBUD=y
# CONFIG_ZSWAP_ZPOOL_DEFAULT_Z3FOLD is not set # CONFIG_ZSWAP_ZPOOL_DEFAULT_ZSMALLOC is not set CONFIG_ZSWAP_ZPOOL_DEFAULT="zbud" CONFIG_ZBUD=y
CONFIG_Z3FOLD=m
CONFIG_ZSMALLOC=y
# CONFIG_ZSMALLOC_STAT is not set CONFIG_ZSMALLOC_CHAIN_SIZE=8

#
# Slab allocator options
#
CONFIG_SLUB=y
# CONFIG_SLUB_TINY is not set
CONFIG_SLAB_MERGE_DEFAULT=y
CONFIG_SLAB_FREELIST_RANDOM=y
CONFIG_SLAB_FREELIST_HARDENED=y
# CONFIG_SLUB_STATS is not set
CONFIG_SLUB_CPU_PARTIAL=y
# CONFIG_RANDOM_KMALLOC_CACHES is not set

# end of Slab allocator options

CONFIG_SHUFFLE_PAGE_ALLOCATOR=y
# CONFIG_COMPAT_BRK is not set
CONFIG_SPARSEMEM=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_ARCH_WANT_OPTIMIZE_DAX_VMEMMAP=y
CONFIG_ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP=y
CONFIG_HAVE_FAST_GUP=y
CONFIG_NUMA_KEEP_MEMINFO=y
CONFIG_MEMORY_ISOLATION=y
CONFIG_EXCLUSIVE_SYSTEM_RAM=y
CONFIG_HAVE_BOOTMEM_INFO_NODE=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y
CONFIG_MEMORY_HOTPLUG=y
CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=y
CONFIG_MEMORY_HOTREMOVE=y
CONFIG_MHP_MEMMAP_ON_MEMORY=y
CONFIG_ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK=y
CONFIG_MEMORY_BALLOON=y
CONFIG_BALLOON_COMPACTION=y
CONFIG_COMPACTION=y
CONFIG_COMPACT_UNEVICTABLE_DEFAULT=1
CONFIG_PAGE_REPORTING=y
CONFIG_MIGRATION=y
CONFIG_DEVICE_MIGRATION=y
CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION=y
CONFIG_ARCH_ENABLE_THP_MIGRATION=y
CONFIG_CONTIG_ALLOC=y
CONFIG_PCP_BATCH_SCALE_MAX=5
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_MMU_NOTIFIER=y
CONFIG_KSM=y
CONFIG_DEFAULT_MMAP_MIN_ADDR=65536
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
CONFIG_MEMORY_FAILURE=y
CONFIG_HWPOISON_INJECT=m
CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
CONFIG_ARCH_WANTS_THP_SWAP=y
CONFIG_TRANSPARENT_HUGEPAGE=y
# CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
# CONFIG_TRANSPARENT_HUGEPAGE_NEVER is not set
CONFIG_THP_SWAP=y
# CONFIG_READ_ONLY_THP_FOR_FS is not set
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_USE_PERCPU_NUMA_NODE_ID=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
# CONFIG_CMA is not set
CONFIG_MEM_SOFT_DIRTY=y
CONFIG_GENERIC_EARLY_IOREMAP=y
# CONFIG_DEFERRED_STRUCT_PAGE_INIT is not set
CONFIG_PAGE_IDLE_FLAG=y
CONFIG_IDLE_PAGE_TRACKING=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_ARCH_HAS_CURRENT_STACK_POINTER=y
CONFIG_ARCH_HAS_PTE_DEVMAP=y
CONFIG_ARCH_HAS_ZONE_DMA_SET=y
CONFIG_ZONE_DMA=y
CONFIG_ZONE_DMA32=y
CONFIG_ZONE_DEVICE=y
CONFIG_HMM_MIRROR=y
CONFIG_GET_FREE_REGION=y
CONFIG_DEVICE_PRIVATE=y
CONFIG_VMAP_PFN=y
CONFIG_ARCH_USES_HIGH_VMA_FLAGS=y
CONFIG_ARCH_HAS_PKEYS=y
CONFIG_VM_EVENT_COUNTERS=y
# CONFIG_PERCPU_STATS is not set
# CONFIG_GUP_TEST is not set
# CONFIG_DMAPOOL_TEST is not set
CONFIG_ARCH_HAS_PTE_SPECIAL=y
CONFIG_MAPPING_DIRTY_HELPERS=y
CONFIG_MEMFD_CREATE=y
CONFIG_SECRETMEM=y
# CONFIG_ANON_VMA_NAME is not set
CONFIG_HAVE_ARCH_USERFAULTFD_WP=y
CONFIG_HAVE_ARCH_USERFAULTFD_MINOR=y
CONFIG_USERFAULTFD=y
CONFIG_PTE_MARKER_UFFD_WP=y
# CONFIG_LRU_GEN is not set
CONFIG_ARCH_SUPPORTS_PER_VMA_LOCK=y
CONFIG_PER_VMA_LOCK=y
CONFIG_LOCK_MM_AND_FIND_VMA=y
CONFIG_IOMMU_MM_DATA=y

#
# Data Access Monitoring
#
# CONFIG_DAMON is not set

# end of Data Access Monitoring
# end of Memory Management options

CONFIG_NET=y
CONFIG_WANT_COMPAT_NETLINK_MESSAGES=y
CONFIG_COMPAT_NETLINK_MESSAGES=y
CONFIG_NET_INGRESS=y
CONFIG_NET_EGRESS=y
CONFIG_NET_XGRESS=y
CONFIG_NET_REDIRECT=y
CONFIG_SKB_EXTENSIONS=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_DIAG=m
CONFIG_UNIX=y
CONFIG_AF_UNIX_OOB=y
CONFIG_UNIX_DIAG=m
CONFIG_TLS=m
CONFIG_TLS_DEVICE=y
# CONFIG_TLS_TOE is not set
CONFIG_XFRM=y
CONFIG_XFRM_OFFLOAD=y
CONFIG_XFRM_ALGO=m
CONFIG_XFRM_USER=m
CONFIG_XFRM_USER_COMPAT=m
CONFIG_XFRM_INTERFACE=m
# CONFIG_XFRM_SUB_POLICY is not set
# CONFIG_XFRM_MIGRATE is not set
CONFIG_XFRM_STATISTICS=y
CONFIG_XFRM_AH=m
CONFIG_XFRM_ESP=m
CONFIG_XFRM_IPCOMP=m
CONFIG_NET_KEY=m
# CONFIG_NET_KEY_MIGRATE is not set
CONFIG_XFRM_ESPINTCP=y
CONFIG_SMC=m
CONFIG_SMC_DIAG=m
CONFIG_XDP_SOCKETS=y
CONFIG_XDP_SOCKETS_DIAG=m
CONFIG_NET_HANDSHAKE=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_FIB_TRIE_STATS=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
CONFIG_IP_ROUTE_CLASSID=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE_DEMUX=m
CONFIG_NET_IP_TUNNEL=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE_COMMON=y
CONFIG_IP_MROUTE=y
CONFIG_IP_MROUTE_MULTIPLE_TABLES=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
CONFIG_SYN_COOKIES=y
CONFIG_NET_IPVTI=m
CONFIG_NET_UDP_TUNNEL=m
CONFIG_NET_FOU=m
CONFIG_NET_FOU_IP_TUNNELS=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_ESP_OFFLOAD=m
CONFIG_INET_ESPINTCP=y
CONFIG_INET_IPCOMP=m
CONFIG_INET_TABLE_PERTURB_ORDER=16
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_DIAG=m
CONFIG_INET_TCP_DIAG=m
CONFIG_INET_UDP_DIAG=m
CONFIG_INET_RAW_DIAG=m
CONFIG_INET_DIAG_DESTROY=y
CONFIG_TCP_CONG_ADVANCED=y
CONFIG_TCP_CONG_BIC=m
CONFIG_TCP_CONG_CUBIC=y
CONFIG_TCP_CONG_WESTWOOD=m
CONFIG_TCP_CONG_HTCP=m
CONFIG_TCP_CONG_HSTCP=m
CONFIG_TCP_CONG_HYBLA=m
CONFIG_TCP_CONG_VEGAS=m
CONFIG_TCP_CONG_NV=m
CONFIG_TCP_CONG_SCALABLE=m
CONFIG_TCP_CONG_LP=m
CONFIG_TCP_CONG_VENO=m
CONFIG_TCP_CONG_YEAH=m
CONFIG_TCP_CONG_ILLINOIS=m
CONFIG_TCP_CONG_DCTCP=m
CONFIG_TCP_CONG_CDG=m
CONFIG_TCP_CONG_BBR=m
CONFIG_DEFAULT_CUBIC=y
# CONFIG_DEFAULT_RENO is not set
CONFIG_DEFAULT_TCP_CONG="cubic"
CONFIG_TCP_SIGPOOL=y
# CONFIG_TCP_AO is not set
CONFIG_TCP_MD5SIG=y
CONFIG_IPV6=y
CONFIG_IPV6_ROUTER_PREF=y
CONFIG_IPV6_ROUTE_INFO=y
# CONFIG_IPV6_OPTIMISTIC_DAD is not set
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_ESP_OFFLOAD=m
CONFIG_INET6_ESPINTCP=y
CONFIG_INET6_IPCOMP=m
CONFIG_IPV6_MIP6=m
CONFIG_IPV6_ILA=m
CONFIG_INET6_XFRM_TUNNEL=m
CONFIG_INET6_TUNNEL=m
CONFIG_IPV6_VTI=m
CONFIG_IPV6_SIT=m
CONFIG_IPV6_SIT_6RD=y
CONFIG_IPV6_NDISC_NODETYPE=y
CONFIG_IPV6_TUNNEL=m
CONFIG_IPV6_GRE=m
CONFIG_IPV6_FOU=m
CONFIG_IPV6_FOU_TUNNEL=m
CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_IPV6_SUBTREES=y
CONFIG_IPV6_MROUTE=y
CONFIG_IPV6_MROUTE_MULTIPLE_TABLES=y
CONFIG_IPV6_PIMSM_V2=y
CONFIG_IPV6_SEG6_LWTUNNEL=y
CONFIG_IPV6_SEG6_HMAC=y
CONFIG_IPV6_SEG6_BPF=y
# CONFIG_IPV6_RPL_LWTUNNEL is not set
CONFIG_IPV6_IOAM6_LWTUNNEL=y
CONFIG_NETLABEL=y
CONFIG_MPTCP=y
CONFIG_INET_MPTCP_DIAG=m
CONFIG_MPTCP_IPV6=y
CONFIG_NETWORK_SECMARK=y
CONFIG_NET_PTP_CLASSIFY=y
CONFIG_NETWORK_PHY_TIMESTAMPING=y
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=m

#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_INGRESS=y
CONFIG_NETFILTER_EGRESS=y
CONFIG_NETFILTER_SKIP_EGRESS=y
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_FAMILY_BRIDGE=y
CONFIG_NETFILTER_FAMILY_ARP=y
CONFIG_NETFILTER_BPF_LINK=y
CONFIG_NETFILTER_NETLINK_HOOK=m
CONFIG_NETFILTER_NETLINK_ACCT=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NETFILTER_NETLINK_OSF=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_LOG_SYSLOG=m
CONFIG_NETFILTER_CONNCOUNT=m
CONFIG_NF_CONNTRACK_MARK=y
CONFIG_NF_CONNTRACK_SECMARK=y
CONFIG_NF_CONNTRACK_ZONES=y
# CONFIG_NF_CONNTRACK_PROCFS is not set
CONFIG_NF_CONNTRACK_EVENTS=y
CONFIG_NF_CONNTRACK_TIMEOUT=y
CONFIG_NF_CONNTRACK_TIMESTAMP=y
CONFIG_NF_CONNTRACK_LABELS=y
CONFIG_NF_CONNTRACK_OVS=y
CONFIG_NF_CT_PROTO_DCCP=y
CONFIG_NF_CT_PROTO_GRE=y
CONFIG_NF_CT_PROTO_SCTP=y
CONFIG_NF_CT_PROTO_UDPLITE=y
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NF_CONNTRACK_IRC=m
CONFIG_NF_CONNTRACK_BROADCAST=m
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NF_CONNTRACK_SNMP=m
CONFIG_NF_CONNTRACK_PPTP=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
CONFIG_NF_CT_NETLINK=m
CONFIG_NF_CT_NETLINK_TIMEOUT=m
CONFIG_NF_CT_NETLINK_HELPER=m
CONFIG_NETFILTER_NETLINK_GLUE_CT=y
CONFIG_NF_NAT=m
CONFIG_NF_NAT_AMANDA=m
CONFIG_NF_NAT_FTP=m
CONFIG_NF_NAT_IRC=m
CONFIG_NF_NAT_SIP=m
CONFIG_NF_NAT_TFTP=m
CONFIG_NF_NAT_REDIRECT=y
CONFIG_NF_NAT_MASQUERADE=y
CONFIG_NF_NAT_OVS=y
CONFIG_NETFILTER_SYNPROXY=m
CONFIG_NF_TABLES=m
CONFIG_NF_TABLES_INET=y
CONFIG_NF_TABLES_NETDEV=y
CONFIG_NFT_NUMGEN=m
CONFIG_NFT_CT=m
CONFIG_NFT_FLOW_OFFLOAD=m
CONFIG_NFT_CONNLIMIT=m
CONFIG_NFT_LOG=m
CONFIG_NFT_LIMIT=m
CONFIG_NFT_MASQ=m
CONFIG_NFT_REDIR=m
CONFIG_NFT_NAT=m
CONFIG_NFT_TUNNEL=m
CONFIG_NFT_QUEUE=m
CONFIG_NFT_QUOTA=m
CONFIG_NFT_REJECT=m
CONFIG_NFT_REJECT_INET=m
CONFIG_NFT_COMPAT=m
CONFIG_NFT_HASH=m
CONFIG_NFT_FIB=m
CONFIG_NFT_FIB_INET=m
CONFIG_NFT_XFRM=m
CONFIG_NFT_SOCKET=m
CONFIG_NFT_OSF=m
CONFIG_NFT_TPROXY=m
CONFIG_NFT_SYNPROXY=m
CONFIG_NF_DUP_NETDEV=m
CONFIG_NFT_DUP_NETDEV=m
CONFIG_NFT_FWD_NETDEV=m
CONFIG_NFT_FIB_NETDEV=m
CONFIG_NFT_REJECT_NETDEV=m
CONFIG_NF_FLOW_TABLE_INET=m
CONFIG_NF_FLOW_TABLE=m
# CONFIG_NF_FLOW_TABLE_PROCFS is not set
CONFIG_NETFILTER_XTABLES=m
CONFIG_NETFILTER_XTABLES_COMPAT=y

#
# Xtables combined modules
#
CONFIG_NETFILTER_XT_MARK=m
CONFIG_NETFILTER_XT_CONNMARK=m
CONFIG_NETFILTER_XT_SET=m

#
# Xtables targets
#
CONFIG_NETFILTER_XT_TARGET_AUDIT=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_CONNSECMARK=m
CONFIG_NETFILTER_XT_TARGET_CT=m
CONFIG_NETFILTER_XT_TARGET_DSCP=m
CONFIG_NETFILTER_XT_TARGET_HL=m
CONFIG_NETFILTER_XT_TARGET_HMARK=m
CONFIG_NETFILTER_XT_TARGET_IDLETIMER=m
CONFIG_NETFILTER_XT_TARGET_LED=m
CONFIG_NETFILTER_XT_TARGET_LOG=m
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_NAT=m
CONFIG_NETFILTER_XT_TARGET_NETMAP=m
CONFIG_NETFILTER_XT_TARGET_NFLOG=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
# CONFIG_NETFILTER_XT_TARGET_NOTRACK is not set
CONFIG_NETFILTER_XT_TARGET_RATEEST=m
CONFIG_NETFILTER_XT_TARGET_REDIRECT=m
CONFIG_NETFILTER_XT_TARGET_MASQUERADE=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_TRACE=m
CONFIG_NETFILTER_XT_TARGET_SECMARK=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m

#
# Xtables matches
#
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
CONFIG_NETFILTER_XT_MATCH_BPF=m
CONFIG_NETFILTER_XT_MATCH_CGROUP=m
CONFIG_NETFILTER_XT_MATCH_CLUSTER=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNLABEL=m
CONFIG_NETFILTER_XT_MATCH_CONNLIMIT=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_CPU=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_DEVGROUP=m
CONFIG_NETFILTER_XT_MATCH_DSCP=m
CONFIG_NETFILTER_XT_MATCH_ECN=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_HL=m
CONFIG_NETFILTER_XT_MATCH_IPCOMP=m
CONFIG_NETFILTER_XT_MATCH_IPRANGE=m
CONFIG_NETFILTER_XT_MATCH_IPVS=m
CONFIG_NETFILTER_XT_MATCH_L2TP=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
CONFIG_NETFILTER_XT_MATCH_NFACCT=m
CONFIG_NETFILTER_XT_MATCH_OSF=m
CONFIG_NETFILTER_XT_MATCH_OWNER=m
CONFIG_NETFILTER_XT_MATCH_POLICY=m
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
CONFIG_NETFILTER_XT_MATCH_QUOTA=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_NETFILTER_XT_MATCH_TIME=m
CONFIG_NETFILTER_XT_MATCH_U32=m

# end of Core Netfilter Configuration

CONFIG_IP_SET=m
CONFIG_IP_SET_MAX=256
CONFIG_IP_SET_BITMAP_IP=m
CONFIG_IP_SET_BITMAP_IPMAC=m
CONFIG_IP_SET_BITMAP_PORT=m
CONFIG_IP_SET_HASH_IP=m
CONFIG_IP_SET_HASH_IPMARK=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
CONFIG_IP_SET_HASH_IPMAC=m
CONFIG_IP_SET_HASH_MAC=m
CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_HASH_NET=m
CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_IP_VS=m
CONFIG_IP_VS_IPV6=y
# CONFIG_IP_VS_DEBUG is not set
CONFIG_IP_VS_TAB_BITS=12

#
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_AH_ESP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y
CONFIG_IP_VS_PROTO_SCTP=y

#
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_FO=m
CONFIG_IP_VS_OVF=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_MH=m
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m
CONFIG_IP_VS_TWOS=m

#
# IPVS SH scheduler
#
CONFIG_IP_VS_SH_TAB_BITS=8

#
# IPVS MH scheduler
#
CONFIG_IP_VS_MH_TAB_INDEX=12

#
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
CONFIG_IP_VS_NFCT=y
CONFIG_IP_VS_PE_SIP=m

#
# IP: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV4=m
CONFIG_IP_NF_IPTABLES_LEGACY=m
CONFIG_NF_SOCKET_IPV4=m
CONFIG_NF_TPROXY_IPV4=m
CONFIG_NF_TABLES_IPV4=y
CONFIG_NFT_REJECT_IPV4=m
CONFIG_NFT_DUP_IPV4=m
CONFIG_NFT_FIB_IPV4=m
CONFIG_NF_TABLES_ARP=y
CONFIG_NF_DUP_IPV4=m
CONFIG_NF_LOG_ARP=m
CONFIG_NF_LOG_IPV4=m
CONFIG_NF_REJECT_IPV4=m
CONFIG_NF_NAT_SNMP_BASIC=m
CONFIG_NF_NAT_PPTP=m
CONFIG_NF_NAT_H323=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_RPFILTER=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_SYNPROXY=m
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_SECURITY=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_NFT_COMPAT_ARP=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m

# end of IP: Netfilter Configuration

#
# IPv6: Netfilter Configuration
#
CONFIG_IP6_NF_IPTABLES_LEGACY=m
CONFIG_NF_SOCKET_IPV6=m
CONFIG_NF_TPROXY_IPV6=m
CONFIG_NF_TABLES_IPV6=y
CONFIG_NFT_REJECT_IPV6=m
CONFIG_NFT_DUP_IPV6=m
CONFIG_NFT_FIB_IPV6=m
CONFIG_NF_DUP_IPV6=m
CONFIG_NF_REJECT_IPV6=m
CONFIG_NF_LOG_IPV6=m
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_AH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_MATCH_FRAG=m
CONFIG_IP6_NF_MATCH_OPTS=m
CONFIG_IP6_NF_MATCH_HL=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m
CONFIG_IP6_NF_MATCH_MH=m
CONFIG_IP6_NF_MATCH_RPFILTER=m
CONFIG_IP6_NF_MATCH_RT=m
CONFIG_IP6_NF_MATCH_SRH=m
CONFIG_IP6_NF_TARGET_HL=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_REJECT=m
CONFIG_IP6_NF_TARGET_SYNPROXY=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_RAW=m
CONFIG_IP6_NF_SECURITY=m
CONFIG_IP6_NF_NAT=m
CONFIG_IP6_NF_TARGET_MASQUERADE=m
CONFIG_IP6_NF_TARGET_NPT=m

# end of IPv6: Netfilter Configuration

CONFIG_NF_DEFRAG_IPV6=m
CONFIG_NF_TABLES_BRIDGE=m
CONFIG_NFT_BRIDGE_META=m
CONFIG_NFT_BRIDGE_REJECT=m
CONFIG_NF_CONNTRACK_BRIDGE=m
CONFIG_BRIDGE_NF_EBTABLES_LEGACY=m
CONFIG_BRIDGE_NF_EBTABLES=m
CONFIG_BRIDGE_EBT_BROUTE=m
CONFIG_BRIDGE_EBT_T_FILTER=m
CONFIG_BRIDGE_EBT_T_NAT=m
CONFIG_BRIDGE_EBT_802_3=m
CONFIG_BRIDGE_EBT_AMONG=m
CONFIG_BRIDGE_EBT_ARP=m
CONFIG_BRIDGE_EBT_IP=m
CONFIG_BRIDGE_EBT_IP6=m
CONFIG_BRIDGE_EBT_LIMIT=m
CONFIG_BRIDGE_EBT_MARK=m
CONFIG_BRIDGE_EBT_PKTTYPE=m
CONFIG_BRIDGE_EBT_STP=m
CONFIG_BRIDGE_EBT_VLAN=m
CONFIG_BRIDGE_EBT_ARPREPLY=m
CONFIG_BRIDGE_EBT_DNAT=m
CONFIG_BRIDGE_EBT_MARK_T=m
CONFIG_BRIDGE_EBT_REDIRECT=m
CONFIG_BRIDGE_EBT_SNAT=m
CONFIG_BRIDGE_EBT_LOG=m
CONFIG_BRIDGE_EBT_NFLOG=m
CONFIG_IP_DCCP=m
CONFIG_INET_DCCP_DIAG=m

#
# DCCP CCIDs Configuration
#
# CONFIG_IP_DCCP_CCID2_DEBUG is not set
# CONFIG_IP_DCCP_CCID3 is not set

# end of DCCP CCIDs Configuration

#
# DCCP Kernel Hacking
#
# CONFIG_IP_DCCP_DEBUG is not set

# end of DCCP Kernel Hacking

CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_DEFAULT_COOKIE_HMAC_MD5 is not set CONFIG_SCTP_DEFAULT_COOKIE_HMAC_SHA1=y
# CONFIG_SCTP_DEFAULT_COOKIE_HMAC_NONE is not set CONFIG_SCTP_COOKIE_HMAC_MD5=y CONFIG_SCTP_COOKIE_HMAC_SHA1=y CONFIG_INET_SCTP_DIAG=m
CONFIG_RDS=m
CONFIG_RDS_RDMA=m
CONFIG_RDS_TCP=m
# CONFIG_RDS_DEBUG is not set
CONFIG_TIPC=m
CONFIG_TIPC_MEDIA_IB=y CONFIG_TIPC_MEDIA_UDP=y
CONFIG_TIPC_CRYPTO=y
CONFIG_TIPC_DIAG=m
CONFIG_ATM=m
CONFIG_ATM_CLIP=m
# CONFIG_ATM_CLIP_NO_ICMP is not set CONFIG_ATM_LANE=m
CONFIG_ATM_MPOA=m
CONFIG_ATM_BR2684=m
# CONFIG_ATM_BR2684_IPFILTER is not set CONFIG_L2TP=m
CONFIG_L2TP_DEBUGFS=m
CONFIG_L2TP_V3=y
CONFIG_L2TP_IP=m
CONFIG_L2TP_ETH=m
CONFIG_STP=m
CONFIG_GARP=m
CONFIG_MRP=m
CONFIG_BRIDGE=m CONFIG_BRIDGE_IGMP_SNOOPING=y CONFIG_BRIDGE_VLAN_FILTERING=y CONFIG_BRIDGE_MRP=y
CONFIG_BRIDGE_CFM=y
CONFIG_NET_DSA=m
CONFIG_NET_DSA_TAG_NONE=m CONFIG_NET_DSA_TAG_AR9331=m CONFIG_NET_DSA_TAG_BRCM_COMMON=m CONFIG_NET_DSA_TAG_BRCM=m CONFIG_NET_DSA_TAG_BRCM_LEGACY=m CONFIG_NET_DSA_TAG_BRCM_PREPEND=m CONFIG_NET_DSA_TAG_HELLCREEK=m CONFIG_NET_DSA_TAG_GSWIP=m CONFIG_NET_DSA_TAG_DSA_COMMON=m CONFIG_NET_DSA_TAG_DSA=m CONFIG_NET_DSA_TAG_EDSA=m CONFIG_NET_DSA_TAG_MTK=m CONFIG_NET_DSA_TAG_KSZ=m CONFIG_NET_DSA_TAG_OCELOT=m CONFIG_NET_DSA_TAG_OCELOT_8021Q=m CONFIG_NET_DSA_TAG_QCA=m CONFIG_NET_DSA_TAG_RTL4_A=m
# CONFIG_NET_DSA_TAG_RTL8_4 is not set
# CONFIG_NET_DSA_TAG_RZN1_A5PSW is not set CONFIG_NET_DSA_TAG_LAN9303=m CONFIG_NET_DSA_TAG_SJA1105=m CONFIG_NET_DSA_TAG_TRAILER=m CONFIG_NET_DSA_TAG_XRS700X=m CONFIG_VLAN_8021Q=m
CONFIG_VLAN_8021Q_GVRP=y CONFIG_VLAN_8021Q_MVRP=y
CONFIG_LLC=m
CONFIG_LLC2=m
CONFIG_ATALK=m
CONFIG_X25=m
CONFIG_LAPB=m
CONFIG_PHONET=m
CONFIG_6LOWPAN=m
# CONFIG_6LOWPAN_DEBUGFS is not set CONFIG_6LOWPAN_NHC=m CONFIG_6LOWPAN_NHC_DEST=m CONFIG_6LOWPAN_NHC_FRAGMENT=m CONFIG_6LOWPAN_NHC_HOP=m CONFIG_6LOWPAN_NHC_IPV6=m CONFIG_6LOWPAN_NHC_MOBILITY=m CONFIG_6LOWPAN_NHC_ROUTING=m CONFIG_6LOWPAN_NHC_UDP=m
# CONFIG_6LOWPAN_GHC_EXT_HDR_HOP is not set # CONFIG_6LOWPAN_GHC_UDP is not set
# CONFIG_6LOWPAN_GHC_ICMPV6 is not set
# CONFIG_6LOWPAN_GHC_EXT_HDR_DEST is not set # CONFIG_6LOWPAN_GHC_EXT_HDR_FRAG is not set # CONFIG_6LOWPAN_GHC_EXT_HDR_ROUTE is not set CONFIG_IEEE802154=m
# CONFIG_IEEE802154_NL802154_EXPERIMENTAL is not set CONFIG_IEEE802154_SOCKET=m CONFIG_IEEE802154_6LOWPAN=m CONFIG_MAC802154=m
CONFIG_NET_SCHED=y

#
# Queueing/Scheduling
#
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_MULTIQ=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFB=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_CBS=m
CONFIG_NET_SCH_ETF=m
CONFIG_NET_SCH_MQPRIO_LIB=m
CONFIG_NET_SCH_TAPRIO=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_DRR=m
CONFIG_NET_SCH_MQPRIO=m
CONFIG_NET_SCH_SKBPRIO=m
CONFIG_NET_SCH_CHOKE=m
CONFIG_NET_SCH_QFQ=m
CONFIG_NET_SCH_CODEL=m
CONFIG_NET_SCH_FQ_CODEL=m
CONFIG_NET_SCH_CAKE=m
CONFIG_NET_SCH_FQ=m
CONFIG_NET_SCH_HHF=m
CONFIG_NET_SCH_PIE=m
CONFIG_NET_SCH_FQ_PIE=m
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_SCH_PLUG=m
CONFIG_NET_SCH_ETS=m
# CONFIG_NET_SCH_DEFAULT is not set

#
# Classification
#
CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
# CONFIG_CLS_U32_PERF is not set
CONFIG_CLS_U32_MARK=y
CONFIG_NET_CLS_FLOW=m
CONFIG_NET_CLS_CGROUP=m
CONFIG_NET_CLS_BPF=m
CONFIG_NET_CLS_FLOWER=m
CONFIG_NET_CLS_MATCHALL=m
CONFIG_NET_EMATCH=y
CONFIG_NET_EMATCH_STACK=32
CONFIG_NET_EMATCH_CMP=m
CONFIG_NET_EMATCH_NBYTE=m
CONFIG_NET_EMATCH_U32=m
CONFIG_NET_EMATCH_META=m
CONFIG_NET_EMATCH_TEXT=m
CONFIG_NET_EMATCH_CANID=m
CONFIG_NET_EMATCH_IPSET=m
CONFIG_NET_EMATCH_IPT=m
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=m
CONFIG_NET_ACT_GACT=m
CONFIG_GACT_PROB=y
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_ACT_SAMPLE=m
CONFIG_NET_ACT_NAT=m
CONFIG_NET_ACT_PEDIT=m
CONFIG_NET_ACT_SIMP=m
CONFIG_NET_ACT_SKBEDIT=m
CONFIG_NET_ACT_CSUM=m
CONFIG_NET_ACT_MPLS=m
CONFIG_NET_ACT_VLAN=m
CONFIG_NET_ACT_BPF=m
CONFIG_NET_ACT_CONNMARK=m
CONFIG_NET_ACT_CTINFO=m
CONFIG_NET_ACT_SKBMOD=m
# CONFIG_NET_ACT_IFE is not set
CONFIG_NET_ACT_TUNNEL_KEY=m
CONFIG_NET_ACT_CT=m
CONFIG_NET_ACT_GATE=m
CONFIG_NET_TC_SKB_EXT=y
CONFIG_NET_SCH_FIFO=y
CONFIG_DCB=y
CONFIG_DNS_RESOLVER=y
CONFIG_BATMAN_ADV=m
# CONFIG_BATMAN_ADV_BATMAN_V is not set
CONFIG_BATMAN_ADV_BLA=y
CONFIG_BATMAN_ADV_DAT=y
CONFIG_BATMAN_ADV_NC=y
CONFIG_BATMAN_ADV_MCAST=y
# CONFIG_BATMAN_ADV_DEBUG is not set
# CONFIG_BATMAN_ADV_TRACING is not set
CONFIG_OPENVSWITCH=m
CONFIG_OPENVSWITCH_GRE=m
CONFIG_OPENVSWITCH_VXLAN=m
CONFIG_OPENVSWITCH_GENEVE=m
CONFIG_VSOCKETS=m
CONFIG_VSOCKETS_DIAG=m
CONFIG_VSOCKETS_LOOPBACK=m
CONFIG_VMWARE_VMCI_VSOCKETS=m
CONFIG_VIRTIO_VSOCKETS=m
CONFIG_VIRTIO_VSOCKETS_COMMON=m
CONFIG_HYPERV_VSOCKETS=m
CONFIG_NETLINK_DIAG=m
CONFIG_MPLS=y
CONFIG_NET_MPLS_GSO=m
CONFIG_MPLS_ROUTING=m
CONFIG_MPLS_IPTUNNEL=m
CONFIG_NET_NSH=m
CONFIG_HSR=m
CONFIG_NET_SWITCHDEV=y
CONFIG_NET_L3_MASTER_DEV=y
CONFIG_QRTR=m
CONFIG_QRTR_SMD=m
CONFIG_QRTR_TUN=m
CONFIG_QRTR_MHI=m
CONFIG_NET_NCSI=y
CONFIG_NCSI_OEM_CMD_GET_MAC=y
# CONFIG_NCSI_OEM_CMD_KEEP_PHY is not set
CONFIG_PCPU_DEV_REFCNT=y
CONFIG_MAX_SKB_FRAGS=17
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_SOCK_RX_QUEUE_MAPPING=y
CONFIG_XPS=y
CONFIG_CGROUP_NET_PRIO=y
CONFIG_CGROUP_NET_CLASSID=y
CONFIG_NET_RX_BUSY_POLL=y
CONFIG_BQL=y
CONFIG_BPF_STREAM_PARSER=y
CONFIG_NET_FLOW_LIMIT=y

#
# Network testing
#
CONFIG_NET_PKTGEN=m
CONFIG_NET_DROP_MONITOR=y

# end of Network testing
# end of Networking options

CONFIG_HAMRADIO=y

#
# Packet Radio protocols
#
CONFIG_AX25=m
CONFIG_AX25_DAMA_SLAVE=y
CONFIG_NETROM=m
CONFIG_ROSE=m

#
# AX.25 network device drivers
#
CONFIG_MKISS=m
CONFIG_6PACK=m
CONFIG_BPQETHER=m
CONFIG_BAYCOM_SER_FDX=m
CONFIG_BAYCOM_SER_HDX=m
CONFIG_BAYCOM_PAR=m
CONFIG_YAM=m

# end of AX.25 network device drivers

CONFIG_CAN=m
CONFIG_CAN_RAW=m
CONFIG_CAN_BCM=m
CONFIG_CAN_GW=m
CONFIG_CAN_J1939=m
CONFIG_CAN_ISOTP=m
CONFIG_BT=m
CONFIG_BT_BREDR=y
CONFIG_BT_RFCOMM=m
CONFIG_BT_RFCOMM_TTY=y
CONFIG_BT_BNEP=m
CONFIG_BT_BNEP_MC_FILTER=y
CONFIG_BT_BNEP_PROTO_FILTER=y
CONFIG_BT_CMTP=m
CONFIG_BT_HIDP=m
CONFIG_BT_LE=y
CONFIG_BT_LE_L2CAP_ECRED=y
CONFIG_BT_6LOWPAN=m
CONFIG_BT_LEDS=y
CONFIG_BT_MSFTEXT=y
CONFIG_BT_AOSPEXT=y
CONFIG_BT_DEBUGFS=y
# CONFIG_BT_SELFTEST is not set

#
# Bluetooth device drivers
#
CONFIG_BT_INTEL=m
CONFIG_BT_BCM=m
CONFIG_BT_RTL=m
CONFIG_BT_QCA=m
CONFIG_BT_MTK=m
CONFIG_BT_HCIBTUSB=m
CONFIG_BT_HCIBTUSB_AUTOSUSPEND=y
CONFIG_BT_HCIBTUSB_POLL_SYNC=y
CONFIG_BT_HCIBTUSB_BCM=y
CONFIG_BT_HCIBTUSB_MTK=y
CONFIG_BT_HCIBTUSB_RTL=y
CONFIG_BT_HCIBTSDIO=m
CONFIG_BT_HCIUART=m
CONFIG_BT_HCIUART_SERDEV=y
CONFIG_BT_HCIUART_H4=y
CONFIG_BT_HCIUART_NOKIA=m
CONFIG_BT_HCIUART_BCSP=y
CONFIG_BT_HCIUART_ATH3K=y
CONFIG_BT_HCIUART_LL=y
CONFIG_BT_HCIUART_3WIRE=y
CONFIG_BT_HCIUART_INTEL=y
CONFIG_BT_HCIUART_BCM=y
CONFIG_BT_HCIUART_RTL=y
CONFIG_BT_HCIUART_QCA=y
CONFIG_BT_HCIUART_AG6XX=y
CONFIG_BT_HCIUART_MRVL=y
CONFIG_BT_HCIBCM203X=m
# CONFIG_BT_HCIBCM4377 is not set
CONFIG_BT_HCIBPA10X=m
CONFIG_BT_HCIBFUSB=m
CONFIG_BT_HCIDTL1=m
CONFIG_BT_HCIBT3C=m
CONFIG_BT_HCIBLUECARD=m
CONFIG_BT_HCIVHCI=m
CONFIG_BT_MRVL=m
CONFIG_BT_MRVL_SDIO=m
CONFIG_BT_ATH3K=m
CONFIG_BT_MTKSDIO=m
CONFIG_BT_MTKUART=m
CONFIG_BT_HCIRSI=m
CONFIG_BT_VIRTIO=m
# CONFIG_BT_NXPUART is not set

# end of Bluetooth device drivers

CONFIG_AF_RXRPC=m
CONFIG_AF_RXRPC_IPV6=y
# CONFIG_AF_RXRPC_INJECT_LOSS is not set
# CONFIG_AF_RXRPC_INJECT_RX_DELAY is not set
# CONFIG_AF_RXRPC_DEBUG is not set
CONFIG_RXKAD=y
# CONFIG_RXPERF is not set
CONFIG_AF_KCM=m
CONFIG_STREAM_PARSER=y
# CONFIG_MCTP is not set
CONFIG_FIB_RULES=y
CONFIG_WIRELESS=y
CONFIG_WIRELESS_EXT=y
CONFIG_WEXT_CORE=y
CONFIG_WEXT_PROC=y
CONFIG_WEXT_SPY=y
CONFIG_WEXT_PRIV=y
CONFIG_CFG80211=m
# CONFIG_NL80211_TESTMODE is not set
# CONFIG_CFG80211_DEVELOPER_WARNINGS is not set
# CONFIG_CFG80211_CERTIFICATION_ONUS is not set
CONFIG_CFG80211_REQUIRE_SIGNED_REGDB=y
CONFIG_CFG80211_USE_KERNEL_REGDB_KEYS=y
CONFIG_CFG80211_DEFAULT_PS=y
CONFIG_CFG80211_DEBUGFS=y
CONFIG_CFG80211_CRDA_SUPPORT=y
CONFIG_CFG80211_WEXT=y
CONFIG_CFG80211_WEXT_EXPORT=y
CONFIG_LIB80211=m
CONFIG_LIB80211_CRYPT_WEP=m
CONFIG_LIB80211_CRYPT_CCMP=m
CONFIG_LIB80211_CRYPT_TKIP=m
# CONFIG_LIB80211_DEBUG is not set
CONFIG_MAC80211=m
CONFIG_MAC80211_HAS_RC=y
CONFIG_MAC80211_RC_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT_MINSTREL=y
CONFIG_MAC80211_RC_DEFAULT="minstrel_ht"
CONFIG_MAC80211_MESH=y
CONFIG_MAC80211_LEDS=y
CONFIG_MAC80211_DEBUGFS=y
CONFIG_MAC80211_MESSAGE_TRACING=y
# CONFIG_MAC80211_DEBUG_MENU is not set
CONFIG_MAC80211_STA_HASH_MAX_SIZE=0
CONFIG_RFKILL=y
CONFIG_RFKILL_LEDS=y
CONFIG_RFKILL_INPUT=y
CONFIG_RFKILL_GPIO=m
CONFIG_NET_9P=m
CONFIG_NET_9P_FD=m
CONFIG_NET_9P_VIRTIO=m
CONFIG_NET_9P_XEN=m
CONFIG_NET_9P_RDMA=m
# CONFIG_NET_9P_DEBUG is not set
CONFIG_CAIF=m
# CONFIG_CAIF_DEBUG is not set
CONFIG_CAIF_NETDEV=m
CONFIG_CAIF_USB=m
CONFIG_CEPH_LIB=m
# CONFIG_CEPH_LIB_PRETTYDEBUG is not set
CONFIG_CEPH_LIB_USE_DNS_RESOLVER=y
CONFIG_NFC=m
CONFIG_NFC_DIGITAL=m
CONFIG_NFC_NCI=m
CONFIG_NFC_NCI_SPI=m
CONFIG_NFC_NCI_UART=m
CONFIG_NFC_HCI=m
CONFIG_NFC_SHDLC=y

#
# Near Field Communication (NFC) devices
#
CONFIG_NFC_TRF7970A=m
CONFIG_NFC_MEI_PHY=m
CONFIG_NFC_SIM=m
CONFIG_NFC_PORT100=m
CONFIG_NFC_VIRTUAL_NCI=m
CONFIG_NFC_FDP=m
CONFIG_NFC_FDP_I2C=m
CONFIG_NFC_PN544=m
CONFIG_NFC_PN544_I2C=m
CONFIG_NFC_PN544_MEI=m
CONFIG_NFC_PN533=m
CONFIG_NFC_PN533_USB=m
CONFIG_NFC_PN533_I2C=m
CONFIG_NFC_PN532_UART=m
CONFIG_NFC_MICROREAD=m
CONFIG_NFC_MICROREAD_I2C=m
CONFIG_NFC_MICROREAD_MEI=m
CONFIG_NFC_MRVL=m
CONFIG_NFC_MRVL_USB=m
CONFIG_NFC_MRVL_UART=m
CONFIG_NFC_MRVL_I2C=m
CONFIG_NFC_MRVL_SPI=m
CONFIG_NFC_ST21NFCA=m
CONFIG_NFC_ST21NFCA_I2C=m
CONFIG_NFC_ST_NCI=m
CONFIG_NFC_ST_NCI_I2C=m
CONFIG_NFC_ST_NCI_SPI=m
CONFIG_NFC_NXP_NCI=m
CONFIG_NFC_NXP_NCI_I2C=m
CONFIG_NFC_S3FWRN5=m
CONFIG_NFC_S3FWRN5_I2C=m
CONFIG_NFC_S3FWRN82_UART=m
CONFIG_NFC_ST95HF=m

# end of Near Field Communication (NFC) devices

CONFIG_PSAMPLE=m
CONFIG_NET_IFE=m
CONFIG_LWTUNNEL=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_DST_CACHE=y
CONFIG_GRO_CELLS=y
CONFIG_SOCK_VALIDATE_XMIT=y
CONFIG_NET_SELFTESTS=y
CONFIG_NET_SOCK_MSG=y
CONFIG_NET_DEVLINK=y
CONFIG_PAGE_POOL=y
# CONFIG_PAGE_POOL_STATS is not set
CONFIG_FAILOVER=m
CONFIG_ETHTOOL_NETLINK=y

#
# Device Drivers
#
CONFIG_HAVE_EISA=y
CONFIG_EISA=y
CONFIG_EISA_VLB_PRIMING=y
CONFIG_EISA_PCI_EISA=y
CONFIG_EISA_VIRTUAL_ROOT=y
CONFIG_EISA_NAMES=y
CONFIG_HAVE_PCI=y
CONFIG_GENERIC_PCI_IOMAP=y
CONFIG_PCI=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=y
CONFIG_PCIEAER=y
# CONFIG_PCIEAER_INJECT is not set
CONFIG_PCIEAER_CXL=y
# CONFIG_PCIE_ECRC is not set
CONFIG_PCIEASPM=y
CONFIG_PCIEASPM_DEFAULT=y
# CONFIG_PCIEASPM_POWERSAVE is not set
# CONFIG_PCIEASPM_POWER_SUPERSAVE is not set
# CONFIG_PCIEASPM_PERFORMANCE is not set
CONFIG_PCIE_PME=y
CONFIG_PCIE_DPC=y
CONFIG_PCIE_PTM=y
CONFIG_PCIE_EDR=y
CONFIG_PCI_MSI=y
CONFIG_PCI_QUIRKS=y
# CONFIG_PCI_DEBUG is not set
CONFIG_PCI_REALLOC_ENABLE_AUTO=y
CONFIG_PCI_STUB=m
CONFIG_PCI_PF_STUB=m
CONFIG_XEN_PCIDEV_FRONTEND=m
CONFIG_PCI_ATS=y
CONFIG_PCI_DOE=y
CONFIG_PCI_LOCKLESS_CONFIG=y
CONFIG_PCI_IOV=y
CONFIG_PCI_PRI=y
CONFIG_PCI_PASID=y
# CONFIG_PCI_P2PDMA is not set
CONFIG_PCI_LABEL=y
CONFIG_PCI_HYPERV=m
# CONFIG_PCIE_BUS_TUNE_OFF is not set
CONFIG_PCIE_BUS_DEFAULT=y
# CONFIG_PCIE_BUS_SAFE is not set
# CONFIG_PCIE_BUS_PERFORMANCE is not set
# CONFIG_PCIE_BUS_PEER2PEER is not set
CONFIG_VGA_ARB=y
CONFIG_VGA_ARB_MAX_GPUS=16
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_ACPI=y
CONFIG_HOTPLUG_PCI_ACPI_IBM=m
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m
CONFIG_HOTPLUG_PCI_SHPC=y

#
# PCI controller drivers
#
CONFIG_VMD=m
CONFIG_PCI_HYPERV_INTERFACE=m

#
# Cadence-based PCIe controllers
#

# end of Cadence-based PCIe controllers

#
# DesignWare-based PCIe controllers
#
CONFIG_PCIE_DW=y
CONFIG_PCIE_DW_HOST=y
CONFIG_PCIE_DW_EP=y
# CONFIG_PCI_MESON is not set
CONFIG_PCIE_DW_PLAT=y
CONFIG_PCIE_DW_PLAT_HOST=y
CONFIG_PCIE_DW_PLAT_EP=y

# end of DesignWare-based PCIe controllers

#
# Mobiveil-based PCIe controllers
#

# end of Mobiveil-based PCIe controllers
# end of PCI controller drivers

#
# PCI Endpoint
#
CONFIG_PCI_ENDPOINT=y
CONFIG_PCI_ENDPOINT_CONFIGFS=y
# CONFIG_PCI_EPF_TEST is not set
CONFIG_PCI_EPF_NTB=m
# CONFIG_PCI_EPF_VNTB is not set

# end of PCI Endpoint

#
# PCI switch controller drivers
#
CONFIG_PCI_SW_SWITCHTEC=m

# end of PCI switch controller drivers

CONFIG_CXL_BUS=m
CONFIG_CXL_PCI=m
CONFIG_CXL_MEM_RAW_COMMANDS=y
CONFIG_CXL_ACPI=m
CONFIG_CXL_PMEM=m
CONFIG_CXL_MEM=m
CONFIG_CXL_PORT=m
CONFIG_CXL_SUSPEND=y
CONFIG_CXL_REGION=y
# CONFIG_CXL_REGION_INVALIDATION_TEST is not set
CONFIG_CXL_PMU=m
CONFIG_PCCARD=m
CONFIG_PCMCIA=m
CONFIG_PCMCIA_LOAD_CIS=y
CONFIG_CARDBUS=y

#
# PC-card bridges
#
CONFIG_YENTA=m
CONFIG_YENTA_O2=y
CONFIG_YENTA_RICOH=y
CONFIG_YENTA_TI=y
CONFIG_YENTA_ENE_TUNE=y
CONFIG_YENTA_TOSHIBA=y
CONFIG_PD6729=m
CONFIG_I82092=m
CONFIG_PCCARD_NONSTATIC=y
CONFIG_RAPIDIO=y
CONFIG_RAPIDIO_TSI721=m
CONFIG_RAPIDIO_DISC_TIMEOUT=30
# CONFIG_RAPIDIO_ENABLE_RX_TX_PORTS is not set
CONFIG_RAPIDIO_DMA_ENGINE=y
# CONFIG_RAPIDIO_DEBUG is not set
CONFIG_RAPIDIO_ENUM_BASIC=m
CONFIG_RAPIDIO_CHMAN=m
CONFIG_RAPIDIO_MPORT_CDEV=m

#
# RapidIO Switch drivers
#
CONFIG_RAPIDIO_CPS_XX=m
CONFIG_RAPIDIO_CPS_GEN2=m
CONFIG_RAPIDIO_RXS_GEN3=m

# end of RapidIO Switch drivers

#
# Generic Driver Options
#
CONFIG_AUXILIARY_BUS=y
CONFIG_UEVENT_HELPER=y
CONFIG_UEVENT_HELPER_PATH=""
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_DEVTMPFS_SAFE is not set
# CONFIG_STANDALONE is not set
CONFIG_PREVENT_FIRMWARE_BUILD=y

#
# Firmware loader
#
CONFIG_FW_LOADER=y CONFIG_FW_LOADER_DEBUG=y CONFIG_FW_LOADER_PAGED_BUF=y CONFIG_FW_LOADER_SYSFS=y CONFIG_EXTRA_FIRMWARE="" CONFIG_FW_LOADER_USER_HELPER=y
# CONFIG_FW_LOADER_USER_HELPER_FALLBACK is not set CONFIG_FW_LOADER_COMPRESS=y CONFIG_FW_LOADER_COMPRESS_XZ=y
# CONFIG_FW_LOADER_COMPRESS_ZSTD is not set CONFIG_FW_CACHE=y
CONFIG_FW_UPLOAD=y

# end of Firmware loader

CONFIG_WANT_DEV_COREDUMP=y
CONFIG_ALLOW_DEV_COREDUMP=y
CONFIG_DEV_COREDUMP=y
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_DEBUG_TEST_DRIVER_REMOVE is not set
CONFIG_HMEM_REPORTING=y
# CONFIG_TEST_ASYNC_DRIVER_PROBE is not set
CONFIG_SYS_HYPERVISOR=y
CONFIG_GENERIC_CPU_DEVICES=y
CONFIG_GENERIC_CPU_AUTOPROBE=y
CONFIG_GENERIC_CPU_VULNERABILITIES=y
CONFIG_REGMAP=y
CONFIG_REGMAP_I2C=y
CONFIG_REGMAP_SLIMBUS=m
CONFIG_REGMAP_SPI=y
CONFIG_REGMAP_SPMI=m
CONFIG_REGMAP_W1=m
CONFIG_REGMAP_MMIO=y
CONFIG_REGMAP_IRQ=y
CONFIG_REGMAP_SOUNDWIRE=m
CONFIG_REGMAP_SOUNDWIRE_MBQ=m
CONFIG_REGMAP_SCCB=m
CONFIG_REGMAP_I3C=m
CONFIG_DMA_SHARED_BUFFER=y
# CONFIG_DMA_FENCE_TRACE is not set
# CONFIG_FW_DEVLINK_SYNC_STATE_TIMEOUT is not set

# end of Generic Driver Options

#
# Bus devices
#
CONFIG_MHI_BUS=m
# CONFIG_MHI_BUS_DEBUG is not set
CONFIG_MHI_BUS_PCI_GENERIC=m
# CONFIG_MHI_BUS_EP is not set

# end of Bus devices

#
# Cache Drivers
#

# end of Cache Drivers

CONFIG_CONNECTOR=y
CONFIG_PROC_EVENTS=y

#
# Firmware Drivers
#

#
# ARM System Control and Management Interface Protocol #

# end of ARM System Control and Management Interface Protocol

CONFIG_EDD=y
CONFIG_EDD_OFF=y
CONFIG_FIRMWARE_MEMMAP=y
CONFIG_DMIID=y
CONFIG_DMI_SYSFS=m
CONFIG_DMI_SCAN_MACHINE_NON_EFI_FALLBACK=y
CONFIG_ISCSI_IBFT_FIND=y
CONFIG_ISCSI_IBFT=m
CONFIG_FW_CFG_SYSFS=m
# CONFIG_FW_CFG_SYSFS_CMDLINE is not set
CONFIG_SYSFB=y
# CONFIG_SYSFB_SIMPLEFB is not set
CONFIG_FW_CS_DSP=m
# CONFIG_GOOGLE_FIRMWARE is not set

#
# EFI (Extensible Firmware Interface) Support #
CONFIG_EFI_ESRT=y
CONFIG_EFI_VARS_PSTORE=m
# CONFIG_EFI_VARS_PSTORE_DEFAULT_DISABLE is not set CONFIG_EFI_SOFT_RESERVE=y CONFIG_EFI_DXE_MEM_ATTRIBUTES=y CONFIG_EFI_RUNTIME_WRAPPERS=y CONFIG_EFI_BOOTLOADER_CONTROL=m CONFIG_EFI_CAPSULE_LOADER=m CONFIG_EFI_TEST=m CONFIG_EFI_DEV_PATH_PARSER=y CONFIG_APPLE_PROPERTIES=y CONFIG_RESET_ATTACK_MITIGATION=y CONFIG_EFI_RCI2_TABLE=y
# CONFIG_EFI_DISABLE_PCI_DMA is not set CONFIG_EFI_EARLYCON=y CONFIG_EFI_CUSTOM_SSDT_OVERLAYS=y
# CONFIG_EFI_DISABLE_RUNTIME is not set
# CONFIG_EFI_COCO_SECRET is not set CONFIG_UNACCEPTED_MEMORY=y CONFIG_EFI_EMBEDDED_FIRMWARE=y

# end of EFI (Extensible Firmware Interface) Support

CONFIG_UEFI_CPER=y
CONFIG_UEFI_CPER_X86=y

#
# Qualcomm firmware drivers
#

# end of Qualcomm firmware drivers

#
# Tegra firmware driver
#

# end of Tegra firmware driver
# end of Firmware Drivers

CONFIG_GNSS=m
CONFIG_GNSS_SERIAL=m
CONFIG_GNSS_MTK_SERIAL=m
CONFIG_GNSS_SIRF_SERIAL=m
CONFIG_GNSS_UBX_SERIAL=m
# CONFIG_GNSS_USB is not set
CONFIG_MTD=m
# CONFIG_MTD_TESTS is not set

#
# Partition parsers
#
CONFIG_MTD_CMDLINE_PARTS=m
CONFIG_MTD_REDBOOT_PARTS=m
CONFIG_MTD_REDBOOT_DIRECTORY_BLOCK=-1
# CONFIG_MTD_REDBOOT_PARTS_UNALLOCATED is not set
# CONFIG_MTD_REDBOOT_PARTS_READONLY is not set

# end of Partition parsers

#
# User Modules And Translation Layers
#
CONFIG_MTD_BLKDEVS=m
CONFIG_MTD_BLOCK=m
CONFIG_MTD_BLOCK_RO=m

#
# Note that in some cases UBI block is preferred. See MTD_UBI_BLOCK. #
CONFIG_FTL=m
CONFIG_NFTL=m
CONFIG_NFTL_RW=y
CONFIG_INFTL=m
CONFIG_RFD_FTL=m
CONFIG_SSFDC=m
CONFIG_SM_FTL=m
CONFIG_MTD_OOPS=m
CONFIG_MTD_PSTORE=m
CONFIG_MTD_SWAP=m
# CONFIG_MTD_PARTITIONED_MASTER is not set

#
# RAM/ROM/Flash chip drivers
#
CONFIG_MTD_CFI=m
CONFIG_MTD_JEDECPROBE=m
CONFIG_MTD_GEN_PROBE=m
# CONFIG_MTD_CFI_ADV_OPTIONS is not set
CONFIG_MTD_MAP_BANK_WIDTH_1=y
CONFIG_MTD_MAP_BANK_WIDTH_2=y
CONFIG_MTD_MAP_BANK_WIDTH_4=y
CONFIG_MTD_CFI_I1=y
CONFIG_MTD_CFI_I2=y
CONFIG_MTD_CFI_INTELEXT=m
CONFIG_MTD_CFI_AMDSTD=m
CONFIG_MTD_CFI_STAA=m
CONFIG_MTD_CFI_UTIL=m
CONFIG_MTD_RAM=m
CONFIG_MTD_ROM=m
CONFIG_MTD_ABSENT=m

# end of RAM/ROM/Flash chip drivers

#
# Mapping drivers for chip access
#
CONFIG_MTD_COMPLEX_MAPPINGS=y
CONFIG_MTD_PHYSMAP=m
# CONFIG_MTD_PHYSMAP_COMPAT is not set
CONFIG_MTD_PHYSMAP_GPIO_ADDR=y
CONFIG_MTD_SBC_GXX=m
CONFIG_MTD_AMD76XROM=m
CONFIG_MTD_ICHXROM=m
CONFIG_MTD_ESB2ROM=m
CONFIG_MTD_CK804XROM=m
CONFIG_MTD_SCB2_FLASH=m
CONFIG_MTD_NETtel=m
CONFIG_MTD_L440GX=m
CONFIG_MTD_PCI=m
CONFIG_MTD_PCMCIA=m
# CONFIG_MTD_PCMCIA_ANONYMOUS is not set
CONFIG_MTD_PLATRAM=m

# end of Mapping drivers for chip access

#
# Self-contained MTD device drivers
#
CONFIG_MTD_PMC551=m
# CONFIG_MTD_PMC551_BUGFIX is not set
# CONFIG_MTD_PMC551_DEBUG is not set
CONFIG_MTD_DATAFLASH=m
# CONFIG_MTD_DATAFLASH_WRITE_VERIFY is not set
CONFIG_MTD_DATAFLASH_OTP=y
CONFIG_MTD_MCHP23K256=m
CONFIG_MTD_MCHP48L640=m
CONFIG_MTD_SST25L=m
CONFIG_MTD_SLRAM=m
CONFIG_MTD_PHRAM=m
CONFIG_MTD_MTDRAM=m
CONFIG_MTDRAM_TOTAL_SIZE=4096
CONFIG_MTDRAM_ERASE_SIZE=128
CONFIG_MTD_BLOCK2MTD=m

#
# Disk-On-Chip Device Drivers
#
# CONFIG_MTD_DOCG3 is not set

# end of Self-contained MTD device drivers

#
# NAND
#
CONFIG_MTD_NAND_CORE=m
CONFIG_MTD_ONENAND=m
CONFIG_MTD_ONENAND_VERIFY_WRITE=y
CONFIG_MTD_ONENAND_GENERIC=m
# CONFIG_MTD_ONENAND_OTP is not set
CONFIG_MTD_ONENAND_2X_PROGRAM=y
CONFIG_MTD_RAW_NAND=m

#
# Raw/parallel NAND flash controllers
#
CONFIG_MTD_NAND_DENALI=m
CONFIG_MTD_NAND_DENALI_PCI=m
CONFIG_MTD_NAND_CAFE=m
CONFIG_MTD_NAND_MXIC=m
CONFIG_MTD_NAND_GPIO=m
CONFIG_MTD_NAND_PLATFORM=m
CONFIG_MTD_NAND_ARASAN=m

#
# Misc
#
CONFIG_MTD_SM_COMMON=m CONFIG_MTD_NAND_NANDSIM=m CONFIG_MTD_NAND_RICOH=m CONFIG_MTD_NAND_DISKONCHIP=m
# CONFIG_MTD_NAND_DISKONCHIP_PROBE_ADVANCED is not set CONFIG_MTD_NAND_DISKONCHIP_PROBE_ADDRESS=0
# CONFIG_MTD_NAND_DISKONCHIP_BBTWRITE is not set CONFIG_MTD_SPI_NAND=m

#
# ECC engine support
#
CONFIG_MTD_NAND_ECC=y
CONFIG_MTD_NAND_ECC_SW_HAMMING=y
# CONFIG_MTD_NAND_ECC_SW_HAMMING_SMC is not set
CONFIG_MTD_NAND_ECC_SW_BCH=y
CONFIG_MTD_NAND_ECC_MXIC=y

# end of ECC engine support
# end of NAND

#
# LPDDR & LPDDR2 PCM memory drivers
#
CONFIG_MTD_LPDDR=m
CONFIG_MTD_QINFO_PROBE=m

# end of LPDDR & LPDDR2 PCM memory drivers

CONFIG_MTD_SPI_NOR=m
CONFIG_MTD_SPI_NOR_USE_4K_SECTORS=y
# CONFIG_MTD_SPI_NOR_SWP_DISABLE is not set
CONFIG_MTD_SPI_NOR_SWP_DISABLE_ON_VOLATILE=y
# CONFIG_MTD_SPI_NOR_SWP_KEEP is not set
CONFIG_MTD_UBI=m
CONFIG_MTD_UBI_WL_THRESHOLD=4096
CONFIG_MTD_UBI_BEB_LIMIT=20
CONFIG_MTD_UBI_FASTMAP=y
CONFIG_MTD_UBI_GLUEBI=m
CONFIG_MTD_UBI_BLOCK=y
# CONFIG_MTD_UBI_NVMEM is not set
CONFIG_MTD_HYPERBUS=m
# CONFIG_OF is not set
CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_SERIAL=m
CONFIG_PARPORT_PC_FIFO=y
# CONFIG_PARPORT_PC_SUPERIO is not set
CONFIG_PARPORT_PC_PCMCIA=m
CONFIG_PARPORT_1284=y
CONFIG_PARPORT_NOT_PC=y
CONFIG_PNP=y
# CONFIG_PNP_DEBUG_MESSAGES is not set

#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_NULL_BLK=m
CONFIG_BLK_DEV_FD=m
# CONFIG_BLK_DEV_FD_RAWCMD is not set
CONFIG_CDROM=y
CONFIG_BLK_DEV_PCIESSD_MTIP32XX=m
CONFIG_ZRAM=m
CONFIG_ZRAM_DEF_COMP_LZORLE=y
# CONFIG_ZRAM_DEF_COMP_ZSTD is not set
# CONFIG_ZRAM_DEF_COMP_LZ4 is not set
# CONFIG_ZRAM_DEF_COMP_LZO is not set
# CONFIG_ZRAM_DEF_COMP_LZ4HC is not set
# CONFIG_ZRAM_DEF_COMP_842 is not set
CONFIG_ZRAM_DEF_COMP="lzo-rle"
CONFIG_ZRAM_WRITEBACK=y
CONFIG_ZRAM_TRACK_ENTRY_ACTIME=y
CONFIG_ZRAM_MEMORY_TRACKING=y
# CONFIG_ZRAM_MULTI_COMP is not set
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_LOOP_MIN_COUNT=8
CONFIG_BLK_DEV_DRBD=m
# CONFIG_DRBD_FAULT_INJECTION is not set
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=m
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=65536
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
CONFIG_ATA_OVER_ETH=m
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_BACKEND=m
CONFIG_VIRTIO_BLK=m
CONFIG_BLK_DEV_RBD=m
# CONFIG_BLK_DEV_UBLK is not set
CONFIG_BLK_DEV_RNBD=y
CONFIG_BLK_DEV_RNBD_CLIENT=m
CONFIG_BLK_DEV_RNBD_SERVER=m

#
# NVME Support
#
CONFIG_NVME_CORE=y
CONFIG_BLK_DEV_NVME=y
CONFIG_NVME_MULTIPATH=y
# CONFIG_NVME_VERBOSE_ERRORS is not set
CONFIG_NVME_HWMON=y
CONFIG_NVME_FABRICS=m
CONFIG_NVME_RDMA=m
CONFIG_NVME_FC=m
CONFIG_NVME_TCP=m
# CONFIG_NVME_TCP_TLS is not set
# CONFIG_NVME_HOST_AUTH is not set
CONFIG_NVME_TARGET=m
CONFIG_NVME_TARGET_PASSTHRU=y
CONFIG_NVME_TARGET_LOOP=m
CONFIG_NVME_TARGET_RDMA=m
CONFIG_NVME_TARGET_FC=m
# CONFIG_NVME_TARGET_FCLOOP is not set
CONFIG_NVME_TARGET_TCP=m
# CONFIG_NVME_TARGET_TCP_TLS is not set
# CONFIG_NVME_TARGET_AUTH is not set

# end of NVME Support

#
# Misc devices
#
CONFIG_SENSORS_LIS3LV02D=m
CONFIG_AD525X_DPOT=m
CONFIG_AD525X_DPOT_I2C=m
CONFIG_AD525X_DPOT_SPI=m
CONFIG_DUMMY_IRQ=m
CONFIG_IBM_ASM=m
CONFIG_PHANTOM=m
CONFIG_TIFM_CORE=m
CONFIG_TIFM_7XX1=m
CONFIG_ICS932S401=m
CONFIG_ENCLOSURE_SERVICES=m
CONFIG_SGI_XP=m
CONFIG_HP_ILO=m
CONFIG_SGI_GRU=m
# CONFIG_SGI_GRU_DEBUG is not set
CONFIG_APDS9802ALS=m
CONFIG_ISL29003=m
CONFIG_ISL29020=m
CONFIG_SENSORS_TSL2550=m
CONFIG_SENSORS_BH1770=m
CONFIG_SENSORS_APDS990X=m
CONFIG_HMC6352=m
CONFIG_DS1682=m
CONFIG_VMWARE_BALLOON=m
CONFIG_LATTICE_ECP3_CONFIG=m
CONFIG_SRAM=y
CONFIG_DW_XDATA_PCIE=m
# CONFIG_PCI_ENDPOINT_TEST is not set
CONFIG_XILINX_SDFEC=m
CONFIG_MISC_RTSX=m
# CONFIG_NSM is not set
CONFIG_C2PORT=m
CONFIG_C2PORT_DURAMAR_2150=m

#
# EEPROM support
#
CONFIG_EEPROM_AT24=m
CONFIG_EEPROM_AT25=m
CONFIG_EEPROM_MAX6875=m
CONFIG_EEPROM_93CX6=m
CONFIG_EEPROM_93XX46=m
CONFIG_EEPROM_IDT_89HPESX=m
CONFIG_EEPROM_EE1004=m

# end of EEPROM support

CONFIG_CB710_CORE=m
# CONFIG_CB710_DEBUG is not set
CONFIG_CB710_DEBUG_ASSUMPTIONS=y

#
# Texas Instruments shared transport line discipline #
CONFIG_TI_ST=m

# end of Texas Instruments shared transport line discipline

CONFIG_SENSORS_LIS3_I2C=m
CONFIG_ALTERA_STAPL=m
CONFIG_INTEL_MEI=m
CONFIG_INTEL_MEI_ME=m
CONFIG_INTEL_MEI_TXE=m
# CONFIG_INTEL_MEI_GSC is not set
# CONFIG_INTEL_MEI_VSC_HW is not set
CONFIG_INTEL_MEI_HDCP=m
# CONFIG_INTEL_MEI_PXP is not set
# CONFIG_INTEL_MEI_GSC_PROXY is not set
CONFIG_VMWARE_VMCI=m
CONFIG_GENWQE=m
CONFIG_GENWQE_PLATFORM_ERROR_RECOVERY=0
CONFIG_ECHO=m
CONFIG_BCM_VK=m
CONFIG_BCM_VK_TTY=y
CONFIG_MISC_ALCOR_PCI=m
CONFIG_MISC_RTSX_PCI=m
CONFIG_MISC_RTSX_USB=m
CONFIG_UACCE=m
CONFIG_PVPANIC=y
CONFIG_PVPANIC_MMIO=m
CONFIG_PVPANIC_PCI=m
# CONFIG_GP_PCI1XXXX is not set

# end of Misc devices

#
# SCSI device support
#
CONFIG_SCSI_MOD=y
CONFIG_RAID_ATTRS=m
CONFIG_SCSI_COMMON=y
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_NETLINK=y
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=m
CONFIG_BLK_DEV_SR=y
CONFIG_CHR_DEV_SG=y
CONFIG_BLK_DEV_BSG=y
CONFIG_CHR_DEV_SCH=m
CONFIG_SCSI_ENCLOSURE=m
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y

#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m
CONFIG_SCSI_ISCSI_ATTRS=m
CONFIG_SCSI_SAS_ATTRS=m
CONFIG_SCSI_SAS_LIBSAS=m
CONFIG_SCSI_SAS_ATA=y
CONFIG_SCSI_SAS_HOST_SMP=y
CONFIG_SCSI_SRP_ATTRS=m

# end of SCSI Transports

CONFIG_SCSI_LOWLEVEL=y
CONFIG_ISCSI_TCP=m
CONFIG_ISCSI_BOOT_SYSFS=m
CONFIG_SCSI_CXGB3_ISCSI=m
CONFIG_SCSI_CXGB4_ISCSI=m
CONFIG_SCSI_BNX2_ISCSI=m
CONFIG_SCSI_BNX2X_FCOE=m
CONFIG_BE2ISCSI=m
CONFIG_BLK_DEV_3W_XXXX_RAID=m
CONFIG_SCSI_HPSA=m
CONFIG_SCSI_3W_9XXX=m
CONFIG_SCSI_3W_SAS=m
CONFIG_SCSI_ACARD=m
CONFIG_SCSI_AHA1740=m
CONFIG_SCSI_AACRAID=m
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=8
CONFIG_AIC7XXX_RESET_DELAY_MS=5000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
CONFIG_AIC7XXX_DEBUG_MASK=0
CONFIG_AIC7XXX_REG_PRETTY_PRINT=y
CONFIG_SCSI_AIC79XX=m
CONFIG_AIC79XX_CMDS_PER_DEVICE=32
CONFIG_AIC79XX_RESET_DELAY_MS=5000
# CONFIG_AIC79XX_DEBUG_ENABLE is not set
CONFIG_AIC79XX_DEBUG_MASK=0
CONFIG_AIC79XX_REG_PRETTY_PRINT=y
CONFIG_SCSI_AIC94XX=m
# CONFIG_AIC94XX_DEBUG is not set
CONFIG_SCSI_MVSAS=m
# CONFIG_SCSI_MVSAS_DEBUG is not set
# CONFIG_SCSI_MVSAS_TASKLET is not set
CONFIG_SCSI_MVUMI=m
CONFIG_SCSI_ADVANSYS=m
CONFIG_SCSI_ARCMSR=m
CONFIG_SCSI_ESAS2R=m
CONFIG_MEGARAID_NEWGEN=y
CONFIG_MEGARAID_MM=m
CONFIG_MEGARAID_MAILBOX=m
CONFIG_MEGARAID_LEGACY=m
CONFIG_MEGARAID_SAS=m
CONFIG_SCSI_MPT3SAS=m
CONFIG_SCSI_MPT2SAS_MAX_SGE=128
CONFIG_SCSI_MPT3SAS_MAX_SGE=128
CONFIG_SCSI_MPT2SAS=m
CONFIG_SCSI_MPI3MR=m
CONFIG_SCSI_SMARTPQI=m
CONFIG_SCSI_HPTIOP=m
CONFIG_SCSI_BUSLOGIC=m
CONFIG_SCSI_FLASHPOINT=y
CONFIG_SCSI_MYRB=m
CONFIG_SCSI_MYRS=m
CONFIG_VMWARE_PVSCSI=m
CONFIG_XEN_SCSI_FRONTEND=m
CONFIG_HYPERV_STORAGE=m
CONFIG_LIBFC=m
CONFIG_LIBFCOE=m
CONFIG_FCOE=m
CONFIG_FCOE_FNIC=m
CONFIG_SCSI_SNIC=m
# CONFIG_SCSI_SNIC_DEBUG_FS is not set
CONFIG_SCSI_DMX3191D=m
CONFIG_SCSI_FDOMAIN=m
CONFIG_SCSI_FDOMAIN_PCI=m
CONFIG_SCSI_ISCI=m
CONFIG_SCSI_IPS=m
CONFIG_SCSI_INITIO=m
CONFIG_SCSI_INIA100=m
CONFIG_SCSI_PPA=m
CONFIG_SCSI_IMM=m
# CONFIG_SCSI_IZIP_SLOW_CTR is not set
CONFIG_SCSI_STEX=m
CONFIG_SCSI_SYM53C8XX_2=m
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
CONFIG_SCSI_SYM53C8XX_MMIO=y
CONFIG_SCSI_IPR=m
CONFIG_SCSI_IPR_TRACE=y
CONFIG_SCSI_IPR_DUMP=y
CONFIG_SCSI_QLOGIC_1280=m
CONFIG_SCSI_QLA_FC=m
CONFIG_TCM_QLA2XXX=m
# CONFIG_TCM_QLA2XXX_DEBUG is not set
CONFIG_SCSI_QLA_ISCSI=m
CONFIG_QEDI=m
CONFIG_QEDF=m
CONFIG_SCSI_LPFC=m
# CONFIG_SCSI_LPFC_DEBUG_FS is not set
CONFIG_SCSI_EFCT=m
CONFIG_SCSI_SIM710=m
CONFIG_SCSI_DC395x=m
CONFIG_SCSI_AM53C974=m
CONFIG_SCSI_WD719X=m
CONFIG_SCSI_DEBUG=m
CONFIG_SCSI_PMCRAID=m
CONFIG_SCSI_PM8001=m
CONFIG_SCSI_BFA_FC=m
CONFIG_SCSI_VIRTIO=m
CONFIG_SCSI_CHELSIO_FCOE=m
CONFIG_SCSI_LOWLEVEL_PCMCIA=y
CONFIG_PCMCIA_AHA152X=m
CONFIG_PCMCIA_FDOMAIN=m
CONFIG_PCMCIA_QLOGIC=m
CONFIG_PCMCIA_SYM53C500=m
CONFIG_SCSI_DH=y
CONFIG_SCSI_DH_RDAC=m
CONFIG_SCSI_DH_HP_SW=m
CONFIG_SCSI_DH_EMC=m
CONFIG_SCSI_DH_ALUA=m

# end of SCSI device support

CONFIG_ATA=y
CONFIG_SATA_HOST=y
CONFIG_PATA_TIMINGS=y
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_FORCE=y
CONFIG_ATA_ACPI=y
CONFIG_SATA_ZPODD=y
CONFIG_SATA_PMP=y

#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=y
CONFIG_SATA_MOBILE_LPM_POLICY=3
CONFIG_SATA_AHCI_PLATFORM=m
# CONFIG_AHCI_DWC is not set
CONFIG_SATA_INIC162X=m
CONFIG_SATA_ACARD_AHCI=m
CONFIG_SATA_SIL24=m
CONFIG_ATA_SFF=y

#
# SFF controllers with custom DMA interface
#
CONFIG_PDC_ADMA=m
CONFIG_SATA_QSTOR=m
CONFIG_SATA_SX4=m
CONFIG_ATA_BMDMA=y

#
# SATA SFF controllers with BMDMA
#
CONFIG_ATA_PIIX=y
CONFIG_SATA_DWC=m
CONFIG_SATA_DWC_OLD_DMA=y
CONFIG_SATA_MV=m
CONFIG_SATA_NV=m
CONFIG_SATA_PROMISE=m
CONFIG_SATA_SIL=m
CONFIG_SATA_SIS=m
CONFIG_SATA_SVW=m
CONFIG_SATA_ULI=m
CONFIG_SATA_VIA=m
CONFIG_SATA_VITESSE=m

#
# PATA SFF controllers with BMDMA
#
CONFIG_PATA_ALI=m
CONFIG_PATA_AMD=m
CONFIG_PATA_ARTOP=m
CONFIG_PATA_ATIIXP=m
CONFIG_PATA_ATP867X=m
CONFIG_PATA_CMD64X=m
CONFIG_PATA_CYPRESS=m
CONFIG_PATA_EFAR=m
CONFIG_PATA_HPT366=m
CONFIG_PATA_HPT37X=m
CONFIG_PATA_HPT3X2N=m
CONFIG_PATA_HPT3X3=m
# CONFIG_PATA_HPT3X3_DMA is not set
CONFIG_PATA_IT8213=m
CONFIG_PATA_IT821X=m
CONFIG_PATA_JMICRON=m
CONFIG_PATA_MARVELL=m
CONFIG_PATA_NETCELL=m
CONFIG_PATA_NINJA32=m
CONFIG_PATA_NS87415=m
CONFIG_PATA_OLDPIIX=m
CONFIG_PATA_OPTIDMA=m
CONFIG_PATA_PDC2027X=m
CONFIG_PATA_PDC_OLD=m
CONFIG_PATA_RADISYS=m
CONFIG_PATA_RDC=m
CONFIG_PATA_SCH=m
CONFIG_PATA_SERVERWORKS=m
CONFIG_PATA_SIL680=m
CONFIG_PATA_SIS=y
CONFIG_PATA_TOSHIBA=m
CONFIG_PATA_TRIFLEX=m
CONFIG_PATA_VIA=m
CONFIG_PATA_WINBOND=m

#
# PIO-only SFF controllers
#
CONFIG_PATA_CMD640_PCI=m
CONFIG_PATA_MPIIX=m
CONFIG_PATA_NS87410=m
CONFIG_PATA_OPTI=m
CONFIG_PATA_PCMCIA=m
CONFIG_PATA_RZ1000=m
# CONFIG_PATA_PARPORT is not set

#
# Generic fallback / legacy drivers
#
CONFIG_PATA_ACPI=m
CONFIG_ATA_GENERIC=y
CONFIG_PATA_LEGACY=m
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_AUTODETECT=y
CONFIG_MD_BITMAP_FILE=y
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID10=m
CONFIG_MD_RAID456=m
CONFIG_MD_CLUSTER=m
CONFIG_BCACHE=m
# CONFIG_BCACHE_DEBUG is not set CONFIG_BCACHE_ASYNC_REGISTRATION=y CONFIG_BLK_DEV_DM_BUILTIN=y
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_DEBUG is not set
CONFIG_DM_BUFIO=m
# CONFIG_DM_DEBUG_BLOCK_MANAGER_LOCKING is not set CONFIG_DM_BIO_PRISON=m
CONFIG_DM_PERSISTENT_DATA=m
CONFIG_DM_UNSTRIPED=m
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_THIN_PROVISIONING=m
CONFIG_DM_CACHE=m
CONFIG_DM_CACHE_SMQ=m
CONFIG_DM_WRITECACHE=m
CONFIG_DM_EBS=m
CONFIG_DM_ERA=m
CONFIG_DM_CLONE=m
CONFIG_DM_MIRROR=m
CONFIG_DM_LOG_USERSPACE=m
CONFIG_DM_RAID=m
CONFIG_DM_ZERO=m
CONFIG_DM_MULTIPATH=m
CONFIG_DM_MULTIPATH_QL=m
CONFIG_DM_MULTIPATH_ST=m
CONFIG_DM_MULTIPATH_HST=m
CONFIG_DM_MULTIPATH_IOA=m
CONFIG_DM_DELAY=m
# CONFIG_DM_DUST is not set
CONFIG_DM_INIT=y
CONFIG_DM_UEVENT=y
CONFIG_DM_FLAKEY=m
CONFIG_DM_VERITY=m CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG=y
# CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG_SECONDARY_KEYRING is not set # CONFIG_DM_VERITY_FEC is not set
CONFIG_DM_SWITCH=m
CONFIG_DM_LOG_WRITES=m
CONFIG_DM_INTEGRITY=m
CONFIG_DM_ZONED=m
CONFIG_DM_AUDIT=y
# CONFIG_DM_VDO is not set
CONFIG_TARGET_CORE=m
CONFIG_TCM_IBLOCK=m
CONFIG_TCM_FILEIO=m
CONFIG_TCM_PSCSI=m
CONFIG_TCM_USER2=m
CONFIG_LOOPBACK_TARGET=m
CONFIG_TCM_FC=m
CONFIG_ISCSI_TARGET=m
CONFIG_ISCSI_TARGET_CXGB4=m
CONFIG_SBP_TARGET=m
# CONFIG_REMOTE_TARGET is not set
CONFIG_FUSION=y
CONFIG_FUSION_SPI=m
CONFIG_FUSION_FC=m
CONFIG_FUSION_SAS=m
CONFIG_FUSION_MAX_SGE=128
CONFIG_FUSION_CTL=m
CONFIG_FUSION_LAN=m
CONFIG_FUSION_LOGGING=y

#
# IEEE 1394 (FireWire) support
#
CONFIG_FIREWIRE=m
CONFIG_FIREWIRE_OHCI=m
CONFIG_FIREWIRE_SBP2=m
CONFIG_FIREWIRE_NET=m
CONFIG_FIREWIRE_NOSY=m

# end of IEEE 1394 (FireWire) support

CONFIG_MACINTOSH_DRIVERS=y
CONFIG_MAC_EMUMOUSEBTN=m
CONFIG_NETDEVICES=y
CONFIG_MII=m
CONFIG_NET_CORE=y
CONFIG_BONDING=m
CONFIG_DUMMY=m
CONFIG_WIREGUARD=m
# CONFIG_WIREGUARD_DEBUG is not set
CONFIG_EQUALIZER=m
CONFIG_NET_FC=y
CONFIG_IFB=m
CONFIG_NET_TEAM=m
CONFIG_NET_TEAM_MODE_BROADCAST=m
CONFIG_NET_TEAM_MODE_ROUNDROBIN=m
CONFIG_NET_TEAM_MODE_RANDOM=m
CONFIG_NET_TEAM_MODE_ACTIVEBACKUP=m
CONFIG_NET_TEAM_MODE_LOADBALANCE=m
CONFIG_MACVLAN=m
CONFIG_MACVTAP=m
CONFIG_IPVLAN_L3S=y
CONFIG_IPVLAN=m
CONFIG_IPVTAP=m
CONFIG_VXLAN=m
CONFIG_GENEVE=m
CONFIG_BAREUDP=m
CONFIG_GTP=m
# CONFIG_AMT is not set
CONFIG_MACSEC=m
CONFIG_NETCONSOLE=m
CONFIG_NETCONSOLE_DYNAMIC=y
# CONFIG_NETCONSOLE_EXTENDED_LOG is not set
CONFIG_NETPOLL=y
CONFIG_NET_POLL_CONTROLLER=y
CONFIG_NTB_NETDEV=m
CONFIG_RIONET=m
CONFIG_RIONET_TX_SIZE=128
CONFIG_RIONET_RX_SIZE=128
CONFIG_TUN=y
CONFIG_TAP=m
# CONFIG_TUN_VNET_CROSS_LE is not set
CONFIG_VETH=m
CONFIG_VIRTIO_NET=m
CONFIG_NLMON=m
# CONFIG_NETKIT is not set
CONFIG_NET_VRF=m
CONFIG_VSOCKMON=m
CONFIG_MHI_NET=m
CONFIG_SUNGEM_PHY=m
CONFIG_ARCNET=m
CONFIG_ARCNET_1201=m
CONFIG_ARCNET_1051=m
CONFIG_ARCNET_RAW=m
CONFIG_ARCNET_CAP=m
CONFIG_ARCNET_COM90xx=m
CONFIG_ARCNET_COM90xxIO=m
CONFIG_ARCNET_RIM_I=m
CONFIG_ARCNET_COM20020=m
CONFIG_ARCNET_COM20020_PCI=m
CONFIG_ARCNET_COM20020_CS=m
CONFIG_ATM_DRIVERS=y
CONFIG_ATM_DUMMY=m
CONFIG_ATM_TCP=m
CONFIG_ATM_LANAI=m
CONFIG_ATM_ENI=m
# CONFIG_ATM_ENI_DEBUG is not set
# CONFIG_ATM_ENI_TUNE_BURST is not set
CONFIG_ATM_NICSTAR=m
# CONFIG_ATM_NICSTAR_USE_SUNI is not set
# CONFIG_ATM_NICSTAR_USE_IDT77105 is not set
CONFIG_ATM_IDT77252=m
# CONFIG_ATM_IDT77252_DEBUG is not set
# CONFIG_ATM_IDT77252_RCV_ALL is not set
CONFIG_ATM_IDT77252_USE_SUNI=y
CONFIG_ATM_IA=m
# CONFIG_ATM_IA_DEBUG is not set
CONFIG_ATM_FORE200E=m
# CONFIG_ATM_FORE200E_USE_TASKLET is not set
CONFIG_ATM_FORE200E_TX_RETRY=16
CONFIG_ATM_FORE200E_DEBUG=0
CONFIG_ATM_HE=m
CONFIG_ATM_HE_USE_SUNI=y
CONFIG_ATM_SOLOS=m
CONFIG_CAIF_DRIVERS=y
CONFIG_CAIF_TTY=m
CONFIG_CAIF_VIRTIO=m

#
# Distributed Switch Architecture drivers
#
CONFIG_B53=m
CONFIG_B53_SPI_DRIVER=m
CONFIG_B53_MDIO_DRIVER=m
CONFIG_B53_MMAP_DRIVER=m
CONFIG_B53_SRAB_DRIVER=m
CONFIG_B53_SERDES=m
CONFIG_NET_DSA_BCM_SF2=m
# CONFIG_NET_DSA_LOOP is not set
CONFIG_NET_DSA_HIRSCHMANN_HELLCREEK=m
CONFIG_NET_DSA_LANTIQ_GSWIP=m
CONFIG_NET_DSA_MT7530=m
CONFIG_NET_DSA_MT7530_MDIO=m
CONFIG_NET_DSA_MT7530_MMIO=m
CONFIG_NET_DSA_MV88E6060=m
CONFIG_NET_DSA_MICROCHIP_KSZ_COMMON=m
CONFIG_NET_DSA_MICROCHIP_KSZ9477_I2C=m
# CONFIG_NET_DSA_MICROCHIP_KSZ_SPI is not set
# CONFIG_NET_DSA_MICROCHIP_KSZ_PTP is not set
CONFIG_NET_DSA_MICROCHIP_KSZ8863_SMI=m
CONFIG_NET_DSA_MV88E6XXX=m
CONFIG_NET_DSA_MV88E6XXX_PTP=y
CONFIG_NET_DSA_MSCC_FELIX_DSA_LIB=m
# CONFIG_NET_DSA_MSCC_OCELOT_EXT is not set
CONFIG_NET_DSA_MSCC_SEVILLE=m
CONFIG_NET_DSA_AR9331=m
CONFIG_NET_DSA_QCA8K=m
# CONFIG_NET_DSA_QCA8K_LEDS_SUPPORT is not set
CONFIG_NET_DSA_SJA1105=m
CONFIG_NET_DSA_SJA1105_PTP=y
CONFIG_NET_DSA_SJA1105_TAS=y
CONFIG_NET_DSA_SJA1105_VL=y
CONFIG_NET_DSA_XRS700X=m
CONFIG_NET_DSA_XRS700X_I2C=m
CONFIG_NET_DSA_XRS700X_MDIO=m
# CONFIG_NET_DSA_REALTEK is not set
CONFIG_NET_DSA_SMSC_LAN9303=m
CONFIG_NET_DSA_SMSC_LAN9303_I2C=m
CONFIG_NET_DSA_SMSC_LAN9303_MDIO=m
CONFIG_NET_DSA_VITESSE_VSC73XX=m
CONFIG_NET_DSA_VITESSE_VSC73XX_SPI=m
CONFIG_NET_DSA_VITESSE_VSC73XX_PLATFORM=m

# end of Distributed Switch Architecture drivers

CONFIG_ETHERNET=y
CONFIG_MDIO=m
CONFIG_NET_VENDOR_3COM=y
CONFIG_EL3=m
CONFIG_PCMCIA_3C574=m
CONFIG_PCMCIA_3C589=m
CONFIG_VORTEX=m
CONFIG_TYPHOON=m
CONFIG_NET_VENDOR_ADAPTEC=y
CONFIG_ADAPTEC_STARFIRE=m
CONFIG_NET_VENDOR_AGERE=y
CONFIG_ET131X=m
CONFIG_NET_VENDOR_ALACRITECH=y
CONFIG_SLICOSS=m
CONFIG_NET_VENDOR_ALTEON=y
CONFIG_ACENIC=m
# CONFIG_ACENIC_OMIT_TIGON_I is not set
CONFIG_ALTERA_TSE=m
CONFIG_NET_VENDOR_AMAZON=y
CONFIG_ENA_ETHERNET=m
CONFIG_NET_VENDOR_AMD=y
CONFIG_AMD8111_ETH=m
CONFIG_PCNET32=m
CONFIG_PCMCIA_NMCLAN=m
CONFIG_AMD_XGBE=m
CONFIG_AMD_XGBE_DCB=y
CONFIG_AMD_XGBE_HAVE_ECC=y
# CONFIG_PDS_CORE is not set
CONFIG_NET_VENDOR_AQUANTIA=y
CONFIG_AQTION=m
CONFIG_NET_VENDOR_ARC=y
CONFIG_NET_VENDOR_ASIX=y
# CONFIG_SPI_AX88796C is not set
CONFIG_NET_VENDOR_ATHEROS=y
CONFIG_ATL2=m
CONFIG_ATL1=m
CONFIG_ATL1E=m
CONFIG_ATL1C=m
CONFIG_ALX=m
CONFIG_CX_ECAT=m
CONFIG_NET_VENDOR_BROADCOM=y
CONFIG_B44=m
CONFIG_B44_PCI_AUTOSELECT=y
CONFIG_B44_PCICORE_AUTOSELECT=y
CONFIG_B44_PCI=y
CONFIG_BCMGENET=m
CONFIG_BNX2=m
CONFIG_CNIC=m
CONFIG_TIGON3=m
CONFIG_TIGON3_HWMON=y
CONFIG_BNX2X=m
CONFIG_BNX2X_SRIOV=y
CONFIG_SYSTEMPORT=m
CONFIG_BNXT=m
CONFIG_BNXT_SRIOV=y
CONFIG_BNXT_FLOWER_OFFLOAD=y
CONFIG_BNXT_DCB=y
CONFIG_BNXT_HWMON=y
CONFIG_NET_VENDOR_CADENCE=y
CONFIG_MACB=m
CONFIG_MACB_USE_HWSTAMP=y
CONFIG_MACB_PCI=m
CONFIG_NET_VENDOR_CAVIUM=y
CONFIG_THUNDER_NIC_PF=m
CONFIG_THUNDER_NIC_VF=m
CONFIG_THUNDER_NIC_BGX=m
CONFIG_THUNDER_NIC_RGX=m
CONFIG_CAVIUM_PTP=m
CONFIG_LIQUIDIO_CORE=m
CONFIG_LIQUIDIO=m
CONFIG_LIQUIDIO_VF=m
CONFIG_NET_VENDOR_CHELSIO=y
CONFIG_CHELSIO_T1=m
CONFIG_CHELSIO_T1_1G=y
CONFIG_CHELSIO_T3=m
CONFIG_CHELSIO_T4=m
CONFIG_CHELSIO_T4_DCB=y
CONFIG_CHELSIO_T4_FCOE=y
CONFIG_CHELSIO_T4VF=m
CONFIG_CHELSIO_LIB=m
CONFIG_CHELSIO_INLINE_CRYPTO=y
CONFIG_CHELSIO_IPSEC_INLINE=m
CONFIG_CHELSIO_TLS_DEVICE=m
CONFIG_NET_VENDOR_CIRRUS=y
CONFIG_NET_VENDOR_CISCO=y
CONFIG_ENIC=m
CONFIG_NET_VENDOR_CORTINA=y
CONFIG_NET_VENDOR_DAVICOM=y
# CONFIG_DM9051 is not set
CONFIG_DNET=m
CONFIG_NET_VENDOR_DEC=y
CONFIG_NET_TULIP=y
CONFIG_DE2104X=m
CONFIG_DE2104X_DSL=0
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_TULIP_NAPI is not set
CONFIG_WINBOND_840=m
CONFIG_DM9102=m
CONFIG_ULI526X=m
CONFIG_PCMCIA_XIRCOM=m
CONFIG_NET_VENDOR_DLINK=y
CONFIG_DL2K=m
CONFIG_SUNDANCE=m
# CONFIG_SUNDANCE_MMIO is not set
CONFIG_NET_VENDOR_EMULEX=y
CONFIG_BE2NET=m
CONFIG_BE2NET_HWMON=y
CONFIG_BE2NET_BE2=y
CONFIG_BE2NET_BE3=y
CONFIG_BE2NET_LANCER=y
CONFIG_BE2NET_SKYHAWK=y
CONFIG_NET_VENDOR_ENGLEDER=y
# CONFIG_TSNEP is not set
CONFIG_NET_VENDOR_EZCHIP=y
CONFIG_NET_VENDOR_FUJITSU=y
CONFIG_PCMCIA_FMVJ18X=m
CONFIG_NET_VENDOR_FUNGIBLE=y
# CONFIG_FUN_ETH is not set
CONFIG_NET_VENDOR_GOOGLE=y
CONFIG_GVE=m
CONFIG_NET_VENDOR_HUAWEI=y
CONFIG_HINIC=m
CONFIG_NET_VENDOR_I825XX=y
CONFIG_NET_VENDOR_INTEL=y
CONFIG_E100=m
CONFIG_E1000=m
CONFIG_E1000E=m
CONFIG_E1000E_HWTS=y
CONFIG_IGB=m
CONFIG_IGB_HWMON=y
CONFIG_IGB_DCA=y
CONFIG_IGBVF=m
CONFIG_IXGBE=m
CONFIG_IXGBE_HWMON=y
CONFIG_IXGBE_DCA=y
CONFIG_IXGBE_DCB=y
CONFIG_IXGBE_IPSEC=y
CONFIG_IXGBEVF=m
CONFIG_IXGBEVF_IPSEC=y
CONFIG_I40E=m
CONFIG_I40E_DCB=y
CONFIG_IAVF=m
CONFIG_I40EVF=m
CONFIG_ICE=m
CONFIG_ICE_HWMON=y
CONFIG_ICE_SWITCHDEV=y
CONFIG_ICE_HWTS=y
CONFIG_FM10K=m
CONFIG_IGC=m
CONFIG_IGC_LEDS=y
# CONFIG_IDPF is not set
CONFIG_JME=m
CONFIG_NET_VENDOR_ADI=y
# CONFIG_ADIN1110 is not set
CONFIG_NET_VENDOR_LITEX=y
CONFIG_NET_VENDOR_MARVELL=y
CONFIG_MVMDIO=m
CONFIG_SKGE=m
# CONFIG_SKGE_DEBUG is not set
CONFIG_SKGE_GENESIS=y
CONFIG_SKY2=m
# CONFIG_SKY2_DEBUG is not set
# CONFIG_OCTEON_EP is not set
# CONFIG_OCTEON_EP_VF is not set
CONFIG_PRESTERA=m
CONFIG_PRESTERA_PCI=m
CONFIG_NET_VENDOR_MELLANOX=y
CONFIG_MLX4_EN=m
CONFIG_MLX4_EN_DCB=y
CONFIG_MLX4_CORE=m
CONFIG_MLX4_DEBUG=y
CONFIG_MLX4_CORE_GEN2=y
CONFIG_MLX5_CORE=m
CONFIG_MLX5_FPGA=y
CONFIG_MLX5_CORE_EN=y
CONFIG_MLX5_EN_ARFS=y
CONFIG_MLX5_EN_RXNFC=y
CONFIG_MLX5_MPFS=y
CONFIG_MLX5_ESWITCH=y
CONFIG_MLX5_BRIDGE=y
CONFIG_MLX5_CLS_ACT=y
CONFIG_MLX5_TC_CT=y
CONFIG_MLX5_TC_SAMPLE=y
CONFIG_MLX5_CORE_EN_DCB=y
CONFIG_MLX5_CORE_IPOIB=y
# CONFIG_MLX5_MACSEC is not set
CONFIG_MLX5_EN_IPSEC=y
CONFIG_MLX5_EN_TLS=y
CONFIG_MLX5_SW_STEERING=y
CONFIG_MLX5_SF=y
CONFIG_MLX5_SF_MANAGER=y
# CONFIG_MLX5_DPLL is not set
CONFIG_MLXSW_CORE=m
CONFIG_MLXSW_CORE_HWMON=y
CONFIG_MLXSW_CORE_THERMAL=y
CONFIG_MLXSW_PCI=m
CONFIG_MLXSW_I2C=m
CONFIG_MLXSW_SPECTRUM=m
CONFIG_MLXSW_SPECTRUM_DCB=y
CONFIG_MLXSW_MINIMAL=m
CONFIG_MLXFW=m
CONFIG_NET_VENDOR_MICREL=y
CONFIG_KS8842=m
CONFIG_KS8851=m
CONFIG_KS8851_MLL=m
CONFIG_KSZ884X_PCI=m
CONFIG_NET_VENDOR_MICROCHIP=y
CONFIG_ENC28J60=m
# CONFIG_ENC28J60_WRITEVERIFY is not set
CONFIG_ENCX24J600=m
CONFIG_LAN743X=m
# CONFIG_VCAP is not set
CONFIG_NET_VENDOR_MICROSEMI=y
CONFIG_MSCC_OCELOT_SWITCH_LIB=m
CONFIG_NET_VENDOR_MICROSOFT=y
CONFIG_MICROSOFT_MANA=m
CONFIG_NET_VENDOR_MYRI=y
CONFIG_MYRI10GE=m
CONFIG_MYRI10GE_DCA=y
CONFIG_FEALNX=m
CONFIG_NET_VENDOR_NI=y
CONFIG_NI_XGE_MANAGEMENT_ENET=m
CONFIG_NET_VENDOR_NATSEMI=y
CONFIG_NATSEMI=m
CONFIG_NS83820=m
CONFIG_NET_VENDOR_NETERION=y
CONFIG_S2IO=m
CONFIG_NET_VENDOR_NETRONOME=y
CONFIG_NFP=m
CONFIG_NFP_APP_FLOWER=y
CONFIG_NFP_APP_ABM_NIC=y
CONFIG_NFP_NET_IPSEC=y
# CONFIG_NFP_DEBUG is not set
CONFIG_NET_VENDOR_8390=y
CONFIG_PCMCIA_AXNET=m
CONFIG_NE2K_PCI=m
CONFIG_PCMCIA_PCNET=m
CONFIG_NET_VENDOR_NVIDIA=y
CONFIG_FORCEDETH=m
CONFIG_NET_VENDOR_OKI=y
CONFIG_ETHOC=m
CONFIG_NET_VENDOR_PACKET_ENGINES=y
CONFIG_HAMACHI=m
CONFIG_YELLOWFIN=m
CONFIG_NET_VENDOR_PENSANDO=y
CONFIG_IONIC=m
CONFIG_NET_VENDOR_QLOGIC=y
CONFIG_QLA3XXX=m
CONFIG_QLCNIC=m
CONFIG_QLCNIC_SRIOV=y
CONFIG_QLCNIC_DCB=y
CONFIG_QLCNIC_HWMON=y
CONFIG_NETXEN_NIC=m
CONFIG_QED=m
CONFIG_QED_LL2=y
CONFIG_QED_SRIOV=y
CONFIG_QEDE=m
CONFIG_QED_RDMA=y
CONFIG_QED_ISCSI=y
CONFIG_QED_FCOE=y
CONFIG_QED_OOO=y
CONFIG_NET_VENDOR_BROCADE=y
CONFIG_BNA=m
CONFIG_NET_VENDOR_QUALCOMM=y
CONFIG_QCOM_EMAC=m
CONFIG_RMNET=m
CONFIG_NET_VENDOR_RDC=y
CONFIG_R6040=m
CONFIG_NET_VENDOR_REALTEK=y
CONFIG_ATP=m
CONFIG_8139CP=m
CONFIG_8139TOO=m
CONFIG_8139TOO_PIO=y
# CONFIG_8139TOO_TUNE_TWISTER is not set
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_R8169=m
CONFIG_R8169_LEDS=y
CONFIG_NET_VENDOR_RENESAS=y
CONFIG_NET_VENDOR_ROCKER=y
CONFIG_ROCKER=m
CONFIG_NET_VENDOR_SAMSUNG=y
CONFIG_SXGBE_ETH=m
CONFIG_NET_VENDOR_SEEQ=y
CONFIG_NET_VENDOR_SILAN=y
CONFIG_SC92031=m
CONFIG_NET_VENDOR_SIS=y
CONFIG_SIS900=m
CONFIG_SIS190=m
CONFIG_NET_VENDOR_SOLARFLARE=y
CONFIG_SFC=m
CONFIG_SFC_MTD=y
CONFIG_SFC_MCDI_MON=y
CONFIG_SFC_SRIOV=y
CONFIG_SFC_MCDI_LOGGING=y
CONFIG_SFC_FALCON=m
CONFIG_SFC_FALCON_MTD=y
# CONFIG_SFC_SIENA is not set
CONFIG_NET_VENDOR_SMSC=y
CONFIG_PCMCIA_SMC91C92=m
CONFIG_EPIC100=m
CONFIG_SMSC911X=m
CONFIG_SMSC9420=m
CONFIG_NET_VENDOR_SOCIONEXT=y
CONFIG_NET_VENDOR_STMICRO=y
CONFIG_STMMAC_ETH=m
# CONFIG_STMMAC_SELFTESTS is not set
CONFIG_STMMAC_PLATFORM=m
CONFIG_DWMAC_GENERIC=m
CONFIG_DWMAC_INTEL=m
CONFIG_STMMAC_PCI=m
CONFIG_NET_VENDOR_SUN=y
CONFIG_HAPPYMEAL=m
CONFIG_SUNGEM=m
CONFIG_CASSINI=m
CONFIG_NIU=m
CONFIG_NET_VENDOR_SYNOPSYS=y
CONFIG_DWC_XLGMAC=m
CONFIG_DWC_XLGMAC_PCI=m
CONFIG_NET_VENDOR_TEHUTI=y
CONFIG_TEHUTI=m
CONFIG_NET_VENDOR_TI=y
# CONFIG_TI_CPSW_PHY_SEL is not set
CONFIG_TLAN=m
CONFIG_NET_VENDOR_VERTEXCOM=y
# CONFIG_MSE102X is not set
CONFIG_NET_VENDOR_VIA=y
CONFIG_VIA_RHINE=m
CONFIG_VIA_RHINE_MMIO=y
CONFIG_VIA_VELOCITY=m
CONFIG_NET_VENDOR_WANGXUN=y
# CONFIG_NGBE is not set
# CONFIG_TXGBE is not set
CONFIG_NET_VENDOR_WIZNET=y
CONFIG_WIZNET_W5100=m
CONFIG_WIZNET_W5300=m
# CONFIG_WIZNET_BUS_DIRECT is not set
# CONFIG_WIZNET_BUS_INDIRECT is not set
CONFIG_WIZNET_BUS_ANY=y
CONFIG_WIZNET_W5100_SPI=m
CONFIG_NET_VENDOR_XILINX=y
CONFIG_XILINX_EMACLITE=m
CONFIG_XILINX_LL_TEMAC=m
CONFIG_NET_VENDOR_XIRCOM=y
CONFIG_PCMCIA_XIRC2PS=m
CONFIG_FDDI=y
CONFIG_DEFXX=m
CONFIG_SKFP=m
# CONFIG_HIPPI is not set
CONFIG_NET_SB1000=m
CONFIG_PHYLINK=m
CONFIG_PHYLIB=y
CONFIG_SWPHY=y
CONFIG_LED_TRIGGER_PHY=y
CONFIG_FIXED_PHY=y
CONFIG_SFP=m

#
# MII PHY device drivers
#
CONFIG_AMD_PHY=m
CONFIG_ADIN_PHY=m
# CONFIG_ADIN1100_PHY is not set
CONFIG_AQUANTIA_PHY=m
CONFIG_AX88796B_PHY=m
CONFIG_BROADCOM_PHY=m
CONFIG_BCM54140_PHY=m
CONFIG_BCM7XXX_PHY=m
CONFIG_BCM84881_PHY=y
CONFIG_BCM87XX_PHY=m
CONFIG_BCM_NET_PHYLIB=m
CONFIG_BCM_NET_PHYPTP=m
CONFIG_CICADA_PHY=m
CONFIG_CORTINA_PHY=m
CONFIG_DAVICOM_PHY=m
CONFIG_ICPLUS_PHY=m
CONFIG_LXT_PHY=m
CONFIG_INTEL_XWAY_PHY=m
CONFIG_LSI_ET1011C_PHY=m
CONFIG_MARVELL_PHY=m
CONFIG_MARVELL_10G_PHY=m
# CONFIG_MARVELL_88Q2XXX_PHY is not set
CONFIG_MARVELL_88X2222_PHY=m
CONFIG_MAXLINEAR_GPHY=m
CONFIG_MEDIATEK_GE_PHY=m
# CONFIG_MEDIATEK_GE_SOC_PHY is not set
CONFIG_MICREL_PHY=m
# CONFIG_MICROCHIP_T1S_PHY is not set
CONFIG_MICROCHIP_PHY=m
CONFIG_MICROCHIP_T1_PHY=m
CONFIG_MICROSEMI_PHY=m
CONFIG_MOTORCOMM_PHY=m
CONFIG_NATIONAL_PHY=m
# CONFIG_NXP_CBTX_PHY is not set
CONFIG_NXP_C45_TJA11XX_PHY=m
CONFIG_NXP_TJA11XX_PHY=m
# CONFIG_NCN26000_PHY is not set
CONFIG_QCOM_NET_PHYLIB=m
CONFIG_AT803X_PHY=m
# CONFIG_QCA83XX_PHY is not set
# CONFIG_QCA808X_PHY is not set
CONFIG_QSEMI_PHY=m
CONFIG_REALTEK_PHY=m
CONFIG_RENESAS_PHY=m
CONFIG_ROCKCHIP_PHY=m
CONFIG_SMSC_PHY=m
CONFIG_STE10XP=m
CONFIG_TERANETICS_PHY=m
CONFIG_DP83822_PHY=m
CONFIG_DP83TC811_PHY=m
CONFIG_DP83848_PHY=m
CONFIG_DP83867_PHY=m
CONFIG_DP83869_PHY=m
# CONFIG_DP83TD510_PHY is not set
# CONFIG_DP83TG720_PHY is not set
CONFIG_VITESSE_PHY=m
CONFIG_XILINX_GMII2RGMII=m
CONFIG_MICREL_KS8995MA=m
# CONFIG_PSE_CONTROLLER is not set
CONFIG_CAN_DEV=m
CONFIG_CAN_VCAN=m
CONFIG_CAN_VXCAN=m
CONFIG_CAN_NETLINK=y
CONFIG_CAN_CALC_BITTIMING=y
CONFIG_CAN_RX_OFFLOAD=y
# CONFIG_CAN_CAN327 is not set
CONFIG_CAN_JANZ_ICAN3=m
CONFIG_CAN_KVASER_PCIEFD=m
CONFIG_CAN_SLCAN=m
CONFIG_CAN_C_CAN=m
CONFIG_CAN_C_CAN_PLATFORM=m
CONFIG_CAN_C_CAN_PCI=m
CONFIG_CAN_CC770=m
CONFIG_CAN_CC770_ISA=m
CONFIG_CAN_CC770_PLATFORM=m
# CONFIG_CAN_CTUCANFD_PCI is not set
# CONFIG_CAN_ESD_402_PCI is not set
CONFIG_CAN_IFI_CANFD=m
CONFIG_CAN_M_CAN=m
CONFIG_CAN_M_CAN_PCI=m
CONFIG_CAN_M_CAN_PLATFORM=m
CONFIG_CAN_M_CAN_TCAN4X5X=m
CONFIG_CAN_PEAK_PCIEFD=m
CONFIG_CAN_SJA1000=m
CONFIG_CAN_EMS_PCI=m
CONFIG_CAN_EMS_PCMCIA=m
CONFIG_CAN_F81601=m
CONFIG_CAN_KVASER_PCI=m
CONFIG_CAN_PEAK_PCI=m
CONFIG_CAN_PEAK_PCIEC=y
CONFIG_CAN_PEAK_PCMCIA=m
CONFIG_CAN_PLX_PCI=m
CONFIG_CAN_SJA1000_ISA=m
CONFIG_CAN_SJA1000_PLATFORM=m
CONFIG_CAN_SOFTING=m
CONFIG_CAN_SOFTING_CS=m

#
# CAN SPI interfaces
#
CONFIG_CAN_HI311X=m
CONFIG_CAN_MCP251X=m
CONFIG_CAN_MCP251XFD=m
# CONFIG_CAN_MCP251XFD_SANITY is not set

# end of CAN SPI interfaces

#
# CAN USB interfaces
#
CONFIG_CAN_8DEV_USB=m
CONFIG_CAN_EMS_USB=m
# CONFIG_CAN_ESD_USB is not set
CONFIG_CAN_ETAS_ES58X=m
# CONFIG_CAN_F81604 is not set
CONFIG_CAN_GS_USB=m
CONFIG_CAN_KVASER_USB=m
CONFIG_CAN_MCBA_USB=m
CONFIG_CAN_PEAK_USB=m
CONFIG_CAN_UCAN=m

# end of CAN USB interfaces

# CONFIG_CAN_DEBUG_DEVICES is not set
CONFIG_MDIO_DEVICE=y
CONFIG_MDIO_BUS=y
CONFIG_FWNODE_MDIO=y
CONFIG_ACPI_MDIO=y
CONFIG_MDIO_DEVRES=y
CONFIG_MDIO_BITBANG=m
CONFIG_MDIO_BCM_UNIMAC=m
CONFIG_MDIO_CAVIUM=m
CONFIG_MDIO_GPIO=m
CONFIG_MDIO_I2C=m
CONFIG_MDIO_MVUSB=m
CONFIG_MDIO_MSCC_MIIM=m
CONFIG_MDIO_REGMAP=m
CONFIG_MDIO_THUNDER=m

#
# MDIO Multiplexers
#

#
# PCS device drivers
#
CONFIG_PCS_XPCS=m
CONFIG_PCS_LYNX=m
CONFIG_PCS_MTK_LYNXI=m

# end of PCS device drivers

CONFIG_PLIP=m
CONFIG_PPP=y
CONFIG_PPP_BSDCOMP=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_FILTER=y
CONFIG_PPP_MPPE=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPPOATM=m
CONFIG_PPPOE=m
# CONFIG_PPPOE_HASH_BITS_1 is not set
# CONFIG_PPPOE_HASH_BITS_2 is not set
CONFIG_PPPOE_HASH_BITS_4=y
# CONFIG_PPPOE_HASH_BITS_8 is not set
CONFIG_PPPOE_HASH_BITS=4
CONFIG_PPTP=m
CONFIG_PPPOL2TP=m
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_SLIP=m
CONFIG_SLHC=y
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLIP_SMART=y
CONFIG_SLIP_MODE_SLIP6=y
CONFIG_USB_NET_DRIVERS=m
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_RTL8152=m
CONFIG_USB_LAN78XX=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_AX88179_178A=m
CONFIG_USB_NET_CDCETHER=m
CONFIG_USB_NET_CDC_EEM=m
CONFIG_USB_NET_CDC_NCM=m
CONFIG_USB_NET_HUAWEI_CDC_NCM=m
CONFIG_USB_NET_CDC_MBIM=m
CONFIG_USB_NET_DM9601=m
CONFIG_USB_NET_SR9700=m
CONFIG_USB_NET_SR9800=m
CONFIG_USB_NET_SMSC75XX=m
CONFIG_USB_NET_SMSC95XX=m
CONFIG_USB_NET_GL620A=m
CONFIG_USB_NET_NET1080=m
CONFIG_USB_NET_PLUSB=m
CONFIG_USB_NET_MCS7830=m
CONFIG_USB_NET_RNDIS_HOST=m
CONFIG_USB_NET_CDC_SUBSET_ENABLE=m
CONFIG_USB_NET_CDC_SUBSET=m
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_KC2190=y
CONFIG_USB_NET_ZAURUS=m
CONFIG_USB_NET_CX82310_ETH=m
CONFIG_USB_NET_KALMIA=m
CONFIG_USB_NET_QMI_WWAN=m
CONFIG_USB_HSO=m
CONFIG_USB_NET_INT51X1=m
CONFIG_USB_CDC_PHONET=m
CONFIG_USB_IPHETH=m
CONFIG_USB_SIERRA_NET=m
CONFIG_USB_VL600=m
CONFIG_USB_NET_CH9200=m
CONFIG_USB_NET_AQC111=m
CONFIG_USB_RTL8153_ECM=m
CONFIG_WLAN=y
CONFIG_WLAN_VENDOR_ADMTEK=y
CONFIG_ADM8211=m
CONFIG_ATH_COMMON=m
CONFIG_WLAN_VENDOR_ATH=y
# CONFIG_ATH_DEBUG is not set
CONFIG_ATH5K=m
# CONFIG_ATH5K_DEBUG is not set
# CONFIG_ATH5K_TRACER is not set
CONFIG_ATH5K_PCI=y
CONFIG_ATH9K_HW=m
CONFIG_ATH9K_COMMON=m
CONFIG_ATH9K_COMMON_DEBUG=y
CONFIG_ATH9K_BTCOEX_SUPPORT=y
CONFIG_ATH9K=m
CONFIG_ATH9K_PCI=y
CONFIG_ATH9K_AHB=y
CONFIG_ATH9K_DEBUGFS=y
CONFIG_ATH9K_STATION_STATISTICS=y
# CONFIG_ATH9K_DYNACK is not set
CONFIG_ATH9K_WOW=y
CONFIG_ATH9K_RFKILL=y
CONFIG_ATH9K_CHANNEL_CONTEXT=y
CONFIG_ATH9K_PCOEM=y
CONFIG_ATH9K_PCI_NO_EEPROM=m
CONFIG_ATH9K_HTC=m
CONFIG_ATH9K_HTC_DEBUGFS=y
CONFIG_ATH9K_HWRNG=y
CONFIG_ATH9K_COMMON_SPECTRAL=y
CONFIG_CARL9170=m
CONFIG_CARL9170_LEDS=y
# CONFIG_CARL9170_DEBUGFS is not set
CONFIG_CARL9170_WPC=y
CONFIG_CARL9170_HWRNG=y
CONFIG_ATH6KL=m
CONFIG_ATH6KL_SDIO=m
CONFIG_ATH6KL_USB=m
# CONFIG_ATH6KL_DEBUG is not set
# CONFIG_ATH6KL_TRACING is not set
CONFIG_AR5523=m
CONFIG_WIL6210=m
CONFIG_WIL6210_ISR_COR=y
CONFIG_WIL6210_TRACING=y
CONFIG_WIL6210_DEBUGFS=y
CONFIG_ATH10K=m
CONFIG_ATH10K_CE=y
CONFIG_ATH10K_PCI=m
CONFIG_ATH10K_SDIO=m
CONFIG_ATH10K_USB=m
# CONFIG_ATH10K_DEBUG is not set
CONFIG_ATH10K_DEBUGFS=y
CONFIG_ATH10K_SPECTRAL=y
CONFIG_ATH10K_TRACING=y
CONFIG_WCN36XX=m
# CONFIG_WCN36XX_DEBUGFS is not set
CONFIG_ATH11K=m
CONFIG_ATH11K_AHB=m
CONFIG_ATH11K_PCI=m
# CONFIG_ATH11K_DEBUG is not set
CONFIG_ATH11K_DEBUGFS=y
CONFIG_ATH11K_TRACING=y
CONFIG_ATH11K_SPECTRAL=y
# CONFIG_ATH12K is not set
CONFIG_WLAN_VENDOR_ATMEL=y
CONFIG_AT76C50X_USB=m
CONFIG_WLAN_VENDOR_BROADCOM=y
CONFIG_B43=m
CONFIG_B43_BCMA=y
CONFIG_B43_SSB=y
CONFIG_B43_BUSES_BCMA_AND_SSB=y
# CONFIG_B43_BUSES_BCMA is not set
# CONFIG_B43_BUSES_SSB is not set
CONFIG_B43_PCI_AUTOSELECT=y
CONFIG_B43_PCICORE_AUTOSELECT=y
# CONFIG_B43_SDIO is not set
CONFIG_B43_BCMA_PIO=y
CONFIG_B43_PIO=y
CONFIG_B43_PHY_G=y
CONFIG_B43_PHY_N=y
CONFIG_B43_PHY_LP=y
CONFIG_B43_PHY_HT=y
CONFIG_B43_LEDS=y
CONFIG_B43_HWRNG=y
# CONFIG_B43_DEBUG is not set
CONFIG_B43LEGACY=m
CONFIG_B43LEGACY_PCI_AUTOSELECT=y
CONFIG_B43LEGACY_PCICORE_AUTOSELECT=y
CONFIG_B43LEGACY_LEDS=y
CONFIG_B43LEGACY_HWRNG=y
# CONFIG_B43LEGACY_DEBUG is not set
CONFIG_B43LEGACY_DMA=y
CONFIG_B43LEGACY_PIO=y
CONFIG_B43LEGACY_DMA_AND_PIO_MODE=y
# CONFIG_B43LEGACY_DMA_MODE is not set
# CONFIG_B43LEGACY_PIO_MODE is not set
CONFIG_BRCMUTIL=m
CONFIG_BRCMSMAC=m
CONFIG_BRCMSMAC_LEDS=y
CONFIG_BRCMFMAC=m
CONFIG_BRCMFMAC_PROTO_BCDC=y
CONFIG_BRCMFMAC_PROTO_MSGBUF=y
CONFIG_BRCMFMAC_SDIO=y
CONFIG_BRCMFMAC_USB=y
CONFIG_BRCMFMAC_PCIE=y
CONFIG_BRCM_TRACING=y
# CONFIG_BRCMDBG is not set
CONFIG_WLAN_VENDOR_INTEL=y
CONFIG_IPW2100=m
CONFIG_IPW2100_MONITOR=y
# CONFIG_IPW2100_DEBUG is not set
CONFIG_IPW2200=m
CONFIG_IPW2200_MONITOR=y
CONFIG_IPW2200_RADIOTAP=y
CONFIG_IPW2200_PROMISCUOUS=y
CONFIG_IPW2200_QOS=y
# CONFIG_IPW2200_DEBUG is not set
CONFIG_LIBIPW=m
# CONFIG_LIBIPW_DEBUG is not set
CONFIG_IWLEGACY=m
CONFIG_IWL4965=m
CONFIG_IWL3945=m

#

# iwl3945 / iwl4965 Debugging Options
#
# CONFIG_IWLEGACY_DEBUG is not set
CONFIG_IWLEGACY_DEBUGFS=y
# end of iwl3945 / iwl4965 Debugging Options

CONFIG_IWLWIFI=m
CONFIG_IWLWIFI_LEDS=y
CONFIG_IWLDVM=m
CONFIG_IWLMVM=m
CONFIG_IWLWIFI_OPMODE_MODULAR=y

#
# Debugging Options
#
# CONFIG_IWLWIFI_DEBUG is not set
CONFIG_IWLWIFI_DEBUGFS=y
CONFIG_IWLWIFI_DEVICE_TRACING=y

# end of Debugging Options

CONFIG_WLAN_VENDOR_INTERSIL=y
CONFIG_P54_COMMON=m
CONFIG_P54_USB=m
CONFIG_P54_PCI=m
CONFIG_P54_SPI=m
# CONFIG_P54_SPI_DEFAULT_EEPROM is not set
CONFIG_P54_LEDS=y
CONFIG_WLAN_VENDOR_MARVELL=y
CONFIG_LIBERTAS=m
CONFIG_LIBERTAS_USB=m
CONFIG_LIBERTAS_SDIO=m
CONFIG_LIBERTAS_SPI=m
# CONFIG_LIBERTAS_DEBUG is not set
CONFIG_LIBERTAS_MESH=y
CONFIG_LIBERTAS_THINFIRM=m
# CONFIG_LIBERTAS_THINFIRM_DEBUG is not set
CONFIG_LIBERTAS_THINFIRM_USB=m
CONFIG_MWIFIEX=m
CONFIG_MWIFIEX_SDIO=m
CONFIG_MWIFIEX_PCIE=m
CONFIG_MWIFIEX_USB=m
CONFIG_MWL8K=m
CONFIG_WLAN_VENDOR_MEDIATEK=y
CONFIG_MT7601U=m
CONFIG_MT76_CORE=m
CONFIG_MT76_LEDS=y
CONFIG_MT76_USB=m
CONFIG_MT76_SDIO=m
CONFIG_MT76x02_LIB=m
CONFIG_MT76x02_USB=m
CONFIG_MT76_CONNAC_LIB=m
CONFIG_MT792x_LIB=m
CONFIG_MT76x0_COMMON=m
CONFIG_MT76x0U=m
CONFIG_MT76x0E=m
CONFIG_MT76x2_COMMON=m
CONFIG_MT76x2E=m
CONFIG_MT76x2U=m
CONFIG_MT7603E=m
CONFIG_MT7615_COMMON=m
CONFIG_MT7615E=m
CONFIG_MT7663_USB_SDIO_COMMON=m
CONFIG_MT7663U=m
CONFIG_MT7663S=m
CONFIG_MT7915E=m
CONFIG_MT7921_COMMON=m
CONFIG_MT7921E=m
# CONFIG_MT7921S is not set
# CONFIG_MT7921U is not set
# CONFIG_MT7996E is not set
# CONFIG_MT7925E is not set
# CONFIG_MT7925U is not set
CONFIG_WLAN_VENDOR_MICROCHIP=y
CONFIG_WILC1000=m
CONFIG_WILC1000_SDIO=m
CONFIG_WILC1000_SPI=m
CONFIG_WILC1000_HW_OOB_INTR=y
CONFIG_WLAN_VENDOR_PURELIFI=y
# CONFIG_PLFXLC is not set
CONFIG_WLAN_VENDOR_RALINK=y
CONFIG_RT2X00=m
CONFIG_RT2400PCI=m
CONFIG_RT2500PCI=m
CONFIG_RT61PCI=m
CONFIG_RT2800PCI=m
CONFIG_RT2800PCI_RT33XX=y
CONFIG_RT2800PCI_RT35XX=y
CONFIG_RT2800PCI_RT53XX=y
CONFIG_RT2800PCI_RT3290=y
CONFIG_RT2500USB=m
CONFIG_RT73USB=m
CONFIG_RT2800USB=m
CONFIG_RT2800USB_RT33XX=y
CONFIG_RT2800USB_RT35XX=y
CONFIG_RT2800USB_RT3573=y
CONFIG_RT2800USB_RT53XX=y
CONFIG_RT2800USB_RT55XX=y
CONFIG_RT2800USB_UNKNOWN=y
CONFIG_RT2800_LIB=m
CONFIG_RT2800_LIB_MMIO=m
CONFIG_RT2X00_LIB_MMIO=m
CONFIG_RT2X00_LIB_PCI=m
CONFIG_RT2X00_LIB_USB=m
CONFIG_RT2X00_LIB=m
CONFIG_RT2X00_LIB_FIRMWARE=y
CONFIG_RT2X00_LIB_CRYPTO=y
CONFIG_RT2X00_LIB_LEDS=y
# CONFIG_RT2X00_LIB_DEBUGFS is not set
# CONFIG_RT2X00_DEBUG is not set
CONFIG_WLAN_VENDOR_REALTEK=y
CONFIG_RTL8180=m
CONFIG_RTL8187=m
CONFIG_RTL8187_LEDS=y
CONFIG_RTL_CARDS=m
CONFIG_RTL8192CE=m
CONFIG_RTL8192SE=m
CONFIG_RTL8192DE=m
CONFIG_RTL8723AE=m
CONFIG_RTL8723BE=m
CONFIG_RTL8188EE=m
CONFIG_RTL8192EE=m
CONFIG_RTL8821AE=m
CONFIG_RTL8192CU=m
CONFIG_RTLWIFI=m
CONFIG_RTLWIFI_PCI=m
CONFIG_RTLWIFI_USB=m
# CONFIG_RTLWIFI_DEBUG is not set
CONFIG_RTL8192C_COMMON=m
CONFIG_RTL8723_COMMON=m
CONFIG_RTLBTCOEXIST=m
CONFIG_RTL8XXXU=m
CONFIG_RTL8XXXU_UNTESTED=y
CONFIG_RTW88=m
CONFIG_RTW88_CORE=m
CONFIG_RTW88_PCI=m
CONFIG_RTW88_8822B=m
CONFIG_RTW88_8822C=m
CONFIG_RTW88_8723D=m
CONFIG_RTW88_8821C=m
CONFIG_RTW88_8822BE=m
# CONFIG_RTW88_8822BS is not set
# CONFIG_RTW88_8822BU is not set
CONFIG_RTW88_8822CE=m
# CONFIG_RTW88_8822CS is not set
# CONFIG_RTW88_8822CU is not set
CONFIG_RTW88_8723DE=m
# CONFIG_RTW88_8723DS is not set
# CONFIG_RTW88_8723DU is not set
CONFIG_RTW88_8821CE=m
# CONFIG_RTW88_8821CS is not set
# CONFIG_RTW88_8821CU is not set
CONFIG_RTW88_DEBUG=y
CONFIG_RTW88_DEBUGFS=y
CONFIG_RTW89=m
CONFIG_RTW89_CORE=m
CONFIG_RTW89_PCI=m
CONFIG_RTW89_8852A=m
# CONFIG_RTW89_8851BE is not set
CONFIG_RTW89_8852AE=m
# CONFIG_RTW89_8852BE is not set
# CONFIG_RTW89_8852CE is not set
CONFIG_RTW89_DEBUG=y
CONFIG_RTW89_DEBUGMSG=y
CONFIG_RTW89_DEBUGFS=y
CONFIG_WLAN_VENDOR_RSI=y
CONFIG_RSI_91X=m
# CONFIG_RSI_DEBUGFS is not set
CONFIG_RSI_SDIO=m
CONFIG_RSI_USB=m
CONFIG_RSI_COEX=y
CONFIG_WLAN_VENDOR_SILABS=y
CONFIG_WFX=m
CONFIG_WLAN_VENDOR_ST=y
CONFIG_CW1200=m
CONFIG_CW1200_WLAN_SDIO=m
CONFIG_CW1200_WLAN_SPI=m
CONFIG_WLAN_VENDOR_TI=y
CONFIG_WL1251=m
CONFIG_WL1251_SPI=m
CONFIG_WL1251_SDIO=m
CONFIG_WL12XX=m
CONFIG_WL18XX=m
CONFIG_WLCORE=m
CONFIG_WLCORE_SDIO=m
CONFIG_WLAN_VENDOR_ZYDAS=y
CONFIG_ZD1211RW=m
# CONFIG_ZD1211RW_DEBUG is not set
CONFIG_WLAN_VENDOR_QUANTENNA=y
CONFIG_QTNFMAC=m
CONFIG_QTNFMAC_PCIE=m
CONFIG_MAC80211_HWSIM=m
CONFIG_VIRT_WIFI=m
CONFIG_WAN=y
CONFIG_HDLC=m
CONFIG_HDLC_RAW=m
CONFIG_HDLC_RAW_ETH=m
CONFIG_HDLC_CISCO=m
CONFIG_HDLC_FR=m
CONFIG_HDLC_PPP=m
CONFIG_HDLC_X25=m
# CONFIG_FRAMER is not set
CONFIG_PCI200SYN=m
CONFIG_WANXL=m
CONFIG_PC300TOO=m
CONFIG_FARSYNC=m
CONFIG_LAPBETHER=m
CONFIG_IEEE802154_DRIVERS=m
CONFIG_IEEE802154_FAKELB=m
CONFIG_IEEE802154_AT86RF230=m
CONFIG_IEEE802154_MRF24J40=m
CONFIG_IEEE802154_CC2520=m
CONFIG_IEEE802154_ATUSB=m
CONFIG_IEEE802154_ADF7242=m
CONFIG_IEEE802154_CA8210=m
CONFIG_IEEE802154_CA8210_DEBUGFS=y
CONFIG_IEEE802154_MCR20A=m
CONFIG_IEEE802154_HWSIM=m

#
# Wireless WAN
#
CONFIG_WWAN=y
CONFIG_WWAN_DEBUGFS=y
CONFIG_WWAN_HWSIM=m
CONFIG_MHI_WWAN_CTRL=m
CONFIG_MHI_WWAN_MBIM=m
CONFIG_RPMSG_WWAN_CTRL=m
CONFIG_IOSM=m
# CONFIG_MTK_T7XX is not set

# end of Wireless WAN

CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_NETDEV_BACKEND=m
CONFIG_VMXNET3=m
CONFIG_FUJITSU_ES=m
CONFIG_USB4_NET=m
CONFIG_HYPERV_NET=m
CONFIG_NETDEVSIM=m
CONFIG_NET_FAILOVER=m
CONFIG_ISDN=y
CONFIG_ISDN_CAPI=y
CONFIG_CAPI_TRACE=y
CONFIG_ISDN_CAPI_MIDDLEWARE=y
CONFIG_MISDN=m
CONFIG_MISDN_DSP=m
CONFIG_MISDN_L1OIP=m

#
# mISDN hardware drivers
#
CONFIG_MISDN_HFCPCI=m
CONFIG_MISDN_HFCMULTI=m
CONFIG_MISDN_HFCUSB=m
CONFIG_MISDN_AVMFRITZ=m
CONFIG_MISDN_SPEEDFAX=m
CONFIG_MISDN_INFINEON=m
CONFIG_MISDN_W6692=m
CONFIG_MISDN_NETJET=m
CONFIG_MISDN_HDLC=m
CONFIG_MISDN_IPAC=m
CONFIG_MISDN_ISAR=m

#
# Input device support
#
CONFIG_INPUT=y
CONFIG_INPUT_LEDS=m
CONFIG_INPUT_FF_MEMLESS=m
CONFIG_INPUT_SPARSEKMAP=m
CONFIG_INPUT_MATRIXKMAP=m
CONFIG_INPUT_VIVALDIFMAP=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
CONFIG_INPUT_EVDEV=y
CONFIG_INPUT_EVBUG=m

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ADC=m
CONFIG_KEYBOARD_ADP5520=m
CONFIG_KEYBOARD_ADP5588=m
CONFIG_KEYBOARD_ADP5589=m
CONFIG_KEYBOARD_APPLESPI=m
CONFIG_KEYBOARD_ATKBD=y
CONFIG_KEYBOARD_QT1050=m
CONFIG_KEYBOARD_QT1070=m
CONFIG_KEYBOARD_QT2160=m
CONFIG_KEYBOARD_DLINK_DIR685=m
CONFIG_KEYBOARD_LKKBD=m
CONFIG_KEYBOARD_GPIO=m
CONFIG_KEYBOARD_GPIO_POLLED=m
CONFIG_KEYBOARD_TCA6416=m
CONFIG_KEYBOARD_TCA8418=m
CONFIG_KEYBOARD_MATRIX=m
CONFIG_KEYBOARD_LM8323=m
CONFIG_KEYBOARD_LM8333=m
CONFIG_KEYBOARD_MAX7359=m
CONFIG_KEYBOARD_MCS=m
CONFIG_KEYBOARD_MPR121=m
CONFIG_KEYBOARD_NEWTON=m
CONFIG_KEYBOARD_OPENCORES=m
# CONFIG_KEYBOARD_PINEPHONE is not set
CONFIG_KEYBOARD_SAMSUNG=m
CONFIG_KEYBOARD_STOWAWAY=m
CONFIG_KEYBOARD_SUNKBD=m
CONFIG_KEYBOARD_IQS62X=m
CONFIG_KEYBOARD_TM2_TOUCHKEY=m
CONFIG_KEYBOARD_TWL4030=m
CONFIG_KEYBOARD_XTKBD=m
CONFIG_KEYBOARD_CROS_EC=m
CONFIG_KEYBOARD_MTK_PMIC=m
# CONFIG_KEYBOARD_CYPRESS_SF is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=m
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_BYD=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_SYNAPTICS_SMBUS=y
CONFIG_MOUSE_PS2_CYPRESS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
CONFIG_MOUSE_PS2_ELANTECH=y
CONFIG_MOUSE_PS2_ELANTECH_SMBUS=y
CONFIG_MOUSE_PS2_SENTELIC=y
CONFIG_MOUSE_PS2_TOUCHKIT=y
CONFIG_MOUSE_PS2_FOCALTECH=y
CONFIG_MOUSE_PS2_VMMOUSE=y
CONFIG_MOUSE_PS2_SMBUS=y
CONFIG_MOUSE_SERIAL=m
CONFIG_MOUSE_APPLETOUCH=m
CONFIG_MOUSE_BCM5974=m
CONFIG_MOUSE_CYAPA=m
CONFIG_MOUSE_ELAN_I2C=m
CONFIG_MOUSE_ELAN_I2C_I2C=y
CONFIG_MOUSE_ELAN_I2C_SMBUS=y
CONFIG_MOUSE_VSXXXAA=m
CONFIG_MOUSE_GPIO=m
CONFIG_MOUSE_SYNAPTICS_I2C=m
CONFIG_MOUSE_SYNAPTICS_USB=m
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=m
CONFIG_JOYSTICK_A3D=m
CONFIG_JOYSTICK_ADC=m
CONFIG_JOYSTICK_ADI=m
CONFIG_JOYSTICK_COBRA=m
CONFIG_JOYSTICK_GF2K=m
CONFIG_JOYSTICK_GRIP=m
CONFIG_JOYSTICK_GRIP_MP=m
CONFIG_JOYSTICK_GUILLEMOT=m
CONFIG_JOYSTICK_INTERACT=m
CONFIG_JOYSTICK_SIDEWINDER=m
CONFIG_JOYSTICK_TMDC=m
CONFIG_JOYSTICK_IFORCE=m
CONFIG_JOYSTICK_IFORCE_USB=m
CONFIG_JOYSTICK_IFORCE_232=m
CONFIG_JOYSTICK_WARRIOR=m
CONFIG_JOYSTICK_MAGELLAN=m
CONFIG_JOYSTICK_SPACEORB=m
CONFIG_JOYSTICK_SPACEBALL=m
CONFIG_JOYSTICK_STINGER=m
CONFIG_JOYSTICK_TWIDJOY=m
CONFIG_JOYSTICK_ZHENHUA=m
CONFIG_JOYSTICK_DB9=m
CONFIG_JOYSTICK_GAMECON=m
CONFIG_JOYSTICK_TURBOGRAFX=m
CONFIG_JOYSTICK_AS5011=m
CONFIG_JOYSTICK_JOYDUMP=m
CONFIG_JOYSTICK_XPAD=m
CONFIG_JOYSTICK_XPAD_FF=y
CONFIG_JOYSTICK_XPAD_LEDS=y
CONFIG_JOYSTICK_WALKERA0701=m
CONFIG_JOYSTICK_PSXPAD_SPI=m
CONFIG_JOYSTICK_PSXPAD_SPI_FF=y
CONFIG_JOYSTICK_PXRC=m
CONFIG_JOYSTICK_QWIIC=m
CONFIG_JOYSTICK_FSIA6B=m
# CONFIG_JOYSTICK_SENSEHAT is not set
# CONFIG_JOYSTICK_SEESAW is not set
CONFIG_INPUT_TABLET=y
CONFIG_TABLET_USB_ACECAD=m
CONFIG_TABLET_USB_AIPTEK=m
CONFIG_TABLET_USB_HANWANG=m
CONFIG_TABLET_USB_KBTAB=m
CONFIG_TABLET_USB_PEGASUS=m
CONFIG_TABLET_SERIAL_WACOM4=m
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_88PM860X=m
CONFIG_TOUCHSCREEN_ADS7846=m
CONFIG_TOUCHSCREEN_AD7877=m
CONFIG_TOUCHSCREEN_AD7879=m
CONFIG_TOUCHSCREEN_AD7879_I2C=m
CONFIG_TOUCHSCREEN_AD7879_SPI=m
CONFIG_TOUCHSCREEN_ADC=m
CONFIG_TOUCHSCREEN_ATMEL_MXT=m
CONFIG_TOUCHSCREEN_ATMEL_MXT_T37=y
CONFIG_TOUCHSCREEN_AUO_PIXCIR=m
CONFIG_TOUCHSCREEN_BU21013=m
CONFIG_TOUCHSCREEN_BU21029=m
CONFIG_TOUCHSCREEN_CHIPONE_ICN8505=m
CONFIG_TOUCHSCREEN_CY8CTMA140=m
CONFIG_TOUCHSCREEN_CY8CTMG110=m
CONFIG_TOUCHSCREEN_CYTTSP_CORE=m
CONFIG_TOUCHSCREEN_CYTTSP_I2C=m
CONFIG_TOUCHSCREEN_CYTTSP_SPI=m
CONFIG_TOUCHSCREEN_CYTTSP4_CORE=m
CONFIG_TOUCHSCREEN_CYTTSP4_I2C=m
CONFIG_TOUCHSCREEN_CYTTSP4_SPI=m
# CONFIG_TOUCHSCREEN_CYTTSP5 is not set
CONFIG_TOUCHSCREEN_DA9034=m
CONFIG_TOUCHSCREEN_DA9052=m
CONFIG_TOUCHSCREEN_DYNAPRO=m
CONFIG_TOUCHSCREEN_HAMPSHIRE=m
CONFIG_TOUCHSCREEN_EETI=m
CONFIG_TOUCHSCREEN_EGALAX_SERIAL=m
CONFIG_TOUCHSCREEN_EXC3000=m
CONFIG_TOUCHSCREEN_FUJITSU=m
CONFIG_TOUCHSCREEN_GOODIX=m
# CONFIG_TOUCHSCREEN_GOODIX_BERLIN_I2C is not set
# CONFIG_TOUCHSCREEN_GOODIX_BERLIN_SPI is not set
CONFIG_TOUCHSCREEN_HIDEEP=m
CONFIG_TOUCHSCREEN_HYCON_HY46XX=m
# CONFIG_TOUCHSCREEN_HYNITRON_CSTXXX is not set
CONFIG_TOUCHSCREEN_ILI210X=m
CONFIG_TOUCHSCREEN_ILITEK=m
CONFIG_TOUCHSCREEN_S6SY761=m
CONFIG_TOUCHSCREEN_GUNZE=m
CONFIG_TOUCHSCREEN_EKTF2127=m
CONFIG_TOUCHSCREEN_ELAN=y
CONFIG_TOUCHSCREEN_ELO=m
CONFIG_TOUCHSCREEN_WACOM_W8001=m
CONFIG_TOUCHSCREEN_WACOM_I2C=m
CONFIG_TOUCHSCREEN_MAX11801=m
CONFIG_TOUCHSCREEN_MCS5000=m
CONFIG_TOUCHSCREEN_MMS114=m
CONFIG_TOUCHSCREEN_MELFAS_MIP4=m
CONFIG_TOUCHSCREEN_MSG2638=m
CONFIG_TOUCHSCREEN_MTOUCH=m
# CONFIG_TOUCHSCREEN_NOVATEK_NVT_TS is not set
# CONFIG_TOUCHSCREEN_IMAGIS is not set
CONFIG_TOUCHSCREEN_INEXIO=m
CONFIG_TOUCHSCREEN_PENMOUNT=m
CONFIG_TOUCHSCREEN_EDT_FT5X06=m
CONFIG_TOUCHSCREEN_TOUCHRIGHT=m
CONFIG_TOUCHSCREEN_TOUCHWIN=m
CONFIG_TOUCHSCREEN_PIXCIR=m
CONFIG_TOUCHSCREEN_WDT87XX_I2C=m
CONFIG_TOUCHSCREEN_WM831X=m
CONFIG_TOUCHSCREEN_WM97XX=m
CONFIG_TOUCHSCREEN_WM9705=y
CONFIG_TOUCHSCREEN_WM9712=y
CONFIG_TOUCHSCREEN_WM9713=y
CONFIG_TOUCHSCREEN_USB_COMPOSITE=m
CONFIG_TOUCHSCREEN_MC13783=m
CONFIG_TOUCHSCREEN_USB_EGALAX=y
CONFIG_TOUCHSCREEN_USB_PANJIT=y
CONFIG_TOUCHSCREEN_USB_3M=y
CONFIG_TOUCHSCREEN_USB_ITM=y
CONFIG_TOUCHSCREEN_USB_ETURBO=y
CONFIG_TOUCHSCREEN_USB_GUNZE=y
CONFIG_TOUCHSCREEN_USB_DMC_TSC10=y
CONFIG_TOUCHSCREEN_USB_IRTOUCH=y
CONFIG_TOUCHSCREEN_USB_IDEALTEK=y
CONFIG_TOUCHSCREEN_USB_GENERAL_TOUCH=y
CONFIG_TOUCHSCREEN_USB_GOTOP=y
CONFIG_TOUCHSCREEN_USB_JASTEC=y
CONFIG_TOUCHSCREEN_USB_ELO=y
CONFIG_TOUCHSCREEN_USB_E2I=y
CONFIG_TOUCHSCREEN_USB_ZYTRONIC=y
CONFIG_TOUCHSCREEN_USB_ETT_TC45USB=y
CONFIG_TOUCHSCREEN_USB_NEXIO=y
CONFIG_TOUCHSCREEN_USB_EASYTOUCH=y
CONFIG_TOUCHSCREEN_TOUCHIT213=m
CONFIG_TOUCHSCREEN_TSC_SERIO=m
CONFIG_TOUCHSCREEN_TSC200X_CORE=m
CONFIG_TOUCHSCREEN_TSC2004=m
CONFIG_TOUCHSCREEN_TSC2005=m
CONFIG_TOUCHSCREEN_TSC2007=m
CONFIG_TOUCHSCREEN_TSC2007_IIO=y
CONFIG_TOUCHSCREEN_PCAP=m
CONFIG_TOUCHSCREEN_RM_TS=m
CONFIG_TOUCHSCREEN_SILEAD=m
CONFIG_TOUCHSCREEN_SIS_I2C=m
CONFIG_TOUCHSCREEN_ST1232=m
CONFIG_TOUCHSCREEN_STMFTS=m
CONFIG_TOUCHSCREEN_SUR40=m
CONFIG_TOUCHSCREEN_SURFACE3_SPI=m
CONFIG_TOUCHSCREEN_SX8654=m
CONFIG_TOUCHSCREEN_TPS6507X=m
CONFIG_TOUCHSCREEN_ZET6223=m
CONFIG_TOUCHSCREEN_ZFORCE=m
# CONFIG_TOUCHSCREEN_COLIBRI_VF50 is not set
CONFIG_TOUCHSCREEN_ROHM_BU21023=m
CONFIG_TOUCHSCREEN_IQS5XX=m
# CONFIG_TOUCHSCREEN_IQS7211 is not set
CONFIG_TOUCHSCREEN_ZINITIX=m
# CONFIG_TOUCHSCREEN_HIMAX_HX83112B is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_88PM860X_ONKEY=m
CONFIG_INPUT_88PM80X_ONKEY=m
CONFIG_INPUT_AD714X=m
CONFIG_INPUT_AD714X_I2C=m
CONFIG_INPUT_AD714X_SPI=m
CONFIG_INPUT_ARIZONA_HAPTICS=m
CONFIG_INPUT_ATC260X_ONKEY=m
CONFIG_INPUT_BMA150=m
CONFIG_INPUT_E3X0_BUTTON=m
CONFIG_INPUT_PCSPKR=m
CONFIG_INPUT_MAX77693_HAPTIC=m
CONFIG_INPUT_MAX8925_ONKEY=m
CONFIG_INPUT_MAX8997_HAPTIC=m
CONFIG_INPUT_MC13783_PWRBUTTON=m
CONFIG_INPUT_MMA8450=m
CONFIG_INPUT_APANEL=m
CONFIG_INPUT_GPIO_BEEPER=m
CONFIG_INPUT_GPIO_DECODER=m
CONFIG_INPUT_GPIO_VIBRA=m
CONFIG_INPUT_ATLAS_BTNS=m
CONFIG_INPUT_ATI_REMOTE2=m
CONFIG_INPUT_KEYSPAN_REMOTE=m
CONFIG_INPUT_KXTJ9=m
CONFIG_INPUT_POWERMATE=m
CONFIG_INPUT_YEALINK=m
CONFIG_INPUT_CM109=m
CONFIG_INPUT_REGULATOR_HAPTIC=m
CONFIG_INPUT_RETU_PWRBUTTON=m
CONFIG_INPUT_AXP20X_PEK=m
CONFIG_INPUT_TWL4030_PWRBUTTON=m
CONFIG_INPUT_TWL4030_VIBRA=m
CONFIG_INPUT_TWL6040_VIBRA=m
CONFIG_INPUT_UINPUT=y
CONFIG_INPUT_PALMAS_PWRBUTTON=m
CONFIG_INPUT_PCF50633_PMU=m
CONFIG_INPUT_PCF8574=m
CONFIG_INPUT_PWM_BEEPER=m
CONFIG_INPUT_PWM_VIBRA=m
CONFIG_INPUT_GPIO_ROTARY_ENCODER=m
CONFIG_INPUT_DA7280_HAPTICS=m
CONFIG_INPUT_DA9052_ONKEY=m
CONFIG_INPUT_DA9055_ONKEY=m
CONFIG_INPUT_DA9063_ONKEY=m
CONFIG_INPUT_WM831X_ON=m
CONFIG_INPUT_PCAP=m
CONFIG_INPUT_ADXL34X=m
CONFIG_INPUT_ADXL34X_I2C=m
CONFIG_INPUT_ADXL34X_SPI=m
CONFIG_INPUT_IMS_PCU=m
CONFIG_INPUT_IQS269A=m
CONFIG_INPUT_IQS626A=m
# CONFIG_INPUT_IQS7222 is not set
CONFIG_INPUT_CMA3000=m
CONFIG_INPUT_CMA3000_I2C=m
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=m
CONFIG_INPUT_IDEAPAD_SLIDEBAR=m
CONFIG_INPUT_SOC_BUTTON_ARRAY=m
CONFIG_INPUT_DRV260X_HAPTICS=m
CONFIG_INPUT_DRV2665_HAPTICS=m
CONFIG_INPUT_DRV2667_HAPTICS=m
CONFIG_INPUT_RAVE_SP_PWRBUTTON=m
CONFIG_RMI4_CORE=m
CONFIG_RMI4_I2C=m
CONFIG_RMI4_SPI=m
CONFIG_RMI4_SMB=m
CONFIG_RMI4_F03=y
CONFIG_RMI4_F03_SERIO=m
CONFIG_RMI4_2D_SENSOR=y
CONFIG_RMI4_F11=y
CONFIG_RMI4_F12=y
CONFIG_RMI4_F30=y
CONFIG_RMI4_F34=y
CONFIG_RMI4_F3A=y
CONFIG_RMI4_F54=y
CONFIG_RMI4_F55=y

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=m
CONFIG_SERIO_CT82C710=m
CONFIG_SERIO_PARKBD=m
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=m
CONFIG_SERIO_ALTERA_PS2=m
CONFIG_SERIO_PS2MULT=m
CONFIG_SERIO_ARC_PS2=m
CONFIG_HYPERV_KEYBOARD=m
CONFIG_SERIO_GPIO_PS2=m
CONFIG_USERIO=m
CONFIG_GAMEPORT=m
CONFIG_GAMEPORT_EMU10K1=m
CONFIG_GAMEPORT_FM801=m

# end of Hardware I/O ports
# end of Input device support

#
# Character devices
#
CONFIG_TTY=y
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_VT_CONSOLE_SLEEP=y
CONFIG_VT_HW_CONSOLE_BINDING=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=0
CONFIG_LEGACY_TIOCSTI=y
CONFIG_LDISC_AUTOLOAD=y

#
# Serial drivers
#
CONFIG_SERIAL_EARLYCON=y CONFIG_SERIAL_8250=y
# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set CONFIG_SERIAL_8250_PNP=y CONFIG_SERIAL_8250_16550A_VARIANTS=y CONFIG_SERIAL_8250_FINTEK=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_DMA=y CONFIG_SERIAL_8250_PCILIB=y CONFIG_SERIAL_8250_PCI=y CONFIG_SERIAL_8250_EXAR=m CONFIG_SERIAL_8250_CS=m CONFIG_SERIAL_8250_MEN_MCB=m CONFIG_SERIAL_8250_NR_UARTS=48 CONFIG_SERIAL_8250_RUNTIME_UARTS=32 CONFIG_SERIAL_8250_EXTENDED=y CONFIG_SERIAL_8250_MANY_PORTS=y
# CONFIG_SERIAL_8250_PCI1XXXX is not set CONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set CONFIG_SERIAL_8250_RSA=y CONFIG_SERIAL_8250_DWLIB=y
# CONFIG_SERIAL_8250_DFL is not set CONFIG_SERIAL_8250_DW=m CONFIG_SERIAL_8250_RT288X=y CONFIG_SERIAL_8250_LPSS=m CONFIG_SERIAL_8250_MID=y CONFIG_SERIAL_8250_PERICOM=y

#
# Non-8250 serial port support
#
CONFIG_SERIAL_KGDB_NMI=y
CONFIG_SERIAL_MAX3100=m
CONFIG_SERIAL_MAX310X=y
CONFIG_SERIAL_UARTLITE=m
CONFIG_SERIAL_UARTLITE_NR_UARTS=1
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_CONSOLE_POLL=y
CONFIG_SERIAL_JSM=m
CONFIG_SERIAL_LANTIQ=m
CONFIG_SERIAL_SCCNXP=y
CONFIG_SERIAL_SCCNXP_CONSOLE=y
CONFIG_SERIAL_SC16IS7XX_CORE=m
CONFIG_SERIAL_SC16IS7XX=m
CONFIG_SERIAL_SC16IS7XX_I2C=y
CONFIG_SERIAL_SC16IS7XX_SPI=y
CONFIG_SERIAL_ALTERA_JTAGUART=m
CONFIG_SERIAL_ALTERA_UART=m
CONFIG_SERIAL_ALTERA_UART_MAXPORTS=4
CONFIG_SERIAL_ALTERA_UART_BAUDRATE=115200
CONFIG_SERIAL_ARC=m
CONFIG_SERIAL_ARC_NR_PORTS=1
CONFIG_SERIAL_RP2=m
CONFIG_SERIAL_RP2_NR_UARTS=32
CONFIG_SERIAL_FSL_LPUART=m
CONFIG_SERIAL_FSL_LINFLEXUART=m
CONFIG_SERIAL_MEN_Z135=m
CONFIG_SERIAL_SPRD=m

# end of Serial drivers

CONFIG_SERIAL_MCTRL_GPIO=y
CONFIG_SERIAL_NONSTANDARD=y
CONFIG_MOXA_INTELLIO=m
CONFIG_MOXA_SMARTIO=m
CONFIG_N_HDLC=m
CONFIG_IPWIRELESS=m
CONFIG_N_GSM=m
CONFIG_NOZOMI=m
CONFIG_NULL_TTY=m
CONFIG_HVC_DRIVER=y
CONFIG_HVC_IRQ=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
# CONFIG_RPMSG_TTY is not set
CONFIG_SERIAL_DEV_BUS=y
CONFIG_SERIAL_DEV_CTRL_TTYPORT=y
CONFIG_TTY_PRINTK=y
CONFIG_TTY_PRINTK_LEVEL=6
CONFIG_PRINTER=m
# CONFIG_LP_CONSOLE is not set
CONFIG_PPDEV=m
CONFIG_VIRTIO_CONSOLE=y
CONFIG_IPMI_HANDLER=m
CONFIG_IPMI_DMI_DECODE=y
CONFIG_IPMI_PLAT_DATA=y
# CONFIG_IPMI_PANIC_EVENT is not set
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_SSIF=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
CONFIG_HW_RANDOM=y
CONFIG_HW_RANDOM_TIMERIOMEM=m
CONFIG_HW_RANDOM_INTEL=m
CONFIG_HW_RANDOM_AMD=m
CONFIG_HW_RANDOM_BA431=m
CONFIG_HW_RANDOM_VIA=m
CONFIG_HW_RANDOM_VIRTIO=m
CONFIG_HW_RANDOM_XIPHERA=m
CONFIG_APPLICOM=m
CONFIG_MWAVE=m
CONFIG_DEVMEM=y
CONFIG_NVRAM=m
CONFIG_DEVPORT=y
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
CONFIG_HPET_MMAP_DEFAULT=y
CONFIG_HANGCHECK_TIMER=m
CONFIG_UV_MMTIMER=m
CONFIG_TCG_TPM=y
CONFIG_HW_RANDOM_TPM=y
CONFIG_TCG_TIS_CORE=y
CONFIG_TCG_TIS=y
CONFIG_TCG_TIS_SPI=m
CONFIG_TCG_TIS_SPI_CR50=y
# CONFIG_TCG_TIS_I2C is not set
CONFIG_TCG_TIS_I2C_CR50=m
CONFIG_TCG_TIS_I2C_ATMEL=m
CONFIG_TCG_TIS_I2C_INFINEON=m
CONFIG_TCG_TIS_I2C_NUVOTON=m
CONFIG_TCG_NSC=m
CONFIG_TCG_ATMEL=m
CONFIG_TCG_INFINEON=m
CONFIG_TCG_XEN=m
CONFIG_TCG_CRB=y
CONFIG_TCG_VTPM_PROXY=m
CONFIG_TCG_TIS_ST33ZP24=m
CONFIG_TCG_TIS_ST33ZP24_I2C=m
CONFIG_TCG_TIS_ST33ZP24_SPI=m
CONFIG_TELCLOCK=m
CONFIG_XILLYBUS_CLASS=m
CONFIG_XILLYBUS=m
CONFIG_XILLYBUS_PCIE=m
CONFIG_XILLYUSB=m

# end of Character devices

#
# I2C support
#
CONFIG_I2C=y
CONFIG_ACPI_I2C_OPREGION=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=y
CONFIG_I2C_MUX=m

#
# Multiplexer I2C Chip support
#
CONFIG_I2C_MUX_GPIO=m
CONFIG_I2C_MUX_LTC4306=m
CONFIG_I2C_MUX_PCA9541=m
CONFIG_I2C_MUX_PCA954x=m
CONFIG_I2C_MUX_REG=m
CONFIG_I2C_MUX_MLXCPLD=m

# end of Multiplexer I2C Chip support

CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_SMBUS=m
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCA=m

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
CONFIG_I2C_CCGX_UCSI=m
CONFIG_I2C_ALI1535=m
CONFIG_I2C_ALI1563=m
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
CONFIG_I2C_AMD756_S4882=m
CONFIG_I2C_AMD8111=m
CONFIG_I2C_AMD_MP2=m
CONFIG_I2C_I801=m
CONFIG_I2C_ISCH=m
CONFIG_I2C_ISMT=m
CONFIG_I2C_PIIX4=y
CONFIG_I2C_CHT_WC=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_NFORCE2_S4985=m
CONFIG_I2C_NVIDIA_GPU=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m

#
# ACPI drivers
#
CONFIG_I2C_SCMI=m

#
# I2C system bus drivers (mostly embedded / system-on-chip) #
CONFIG_I2C_CBUS_GPIO=m CONFIG_I2C_DESIGNWARE_CORE=y
# CONFIG_I2C_DESIGNWARE_SLAVE is not set CONFIG_I2C_DESIGNWARE_PLATFORM=y CONFIG_I2C_DESIGNWARE_BAYTRAIL=y CONFIG_I2C_DESIGNWARE_PCI=m
# CONFIG_I2C_EMEV2 is not set
CONFIG_I2C_GPIO=m
# CONFIG_I2C_GPIO_FAULT_INJECTOR is not set CONFIG_I2C_KEMPLD=m
CONFIG_I2C_OCORES=m
CONFIG_I2C_PCA_PLATFORM=m
CONFIG_I2C_SIMTEC=m
CONFIG_I2C_XILINX=m

#
# External I2C/SMBus adapter drivers
#
CONFIG_I2C_DIOLAN_U2C=m
CONFIG_I2C_DLN2=m
CONFIG_I2C_CP2615=m
CONFIG_I2C_PARPORT=m
# CONFIG_I2C_PCI1XXXX is not set
CONFIG_I2C_ROBOTFUZZ_OSIF=m
CONFIG_I2C_TAOS_EVM=m
CONFIG_I2C_TINY_USB=m
CONFIG_I2C_VIPERBOARD=m

#
# Other I2C/SMBus bus drivers
#
CONFIG_I2C_MLXCPLD=m
CONFIG_I2C_CROS_EC_TUNNEL=m
CONFIG_I2C_VIRTIO=m

# end of I2C Hardware Bus support

CONFIG_I2C_STUB=m
# CONFIG_I2C_SLAVE is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set

# end of I2C support

CONFIG_I3C=m
CONFIG_CDNS_I3C_MASTER=m
CONFIG_DW_I3C_MASTER=m
CONFIG_SVC_I3C_MASTER=m
CONFIG_MIPI_I3C_HCI=m
CONFIG_SPI=y
# CONFIG_SPI_DEBUG is not set
CONFIG_SPI_MASTER=y
CONFIG_SPI_MEM=y

#
# SPI Master Controller Drivers
#
CONFIG_SPI_ALTERA=m
CONFIG_SPI_ALTERA_CORE=m
CONFIG_SPI_ALTERA_DFL=m
CONFIG_SPI_AXI_SPI_ENGINE=m
CONFIG_SPI_BITBANG=m
CONFIG_SPI_BUTTERFLY=m
CONFIG_SPI_CADENCE=m
CONFIG_SPI_DESIGNWARE=m
CONFIG_SPI_DW_DMA=y
CONFIG_SPI_DW_PCI=m
CONFIG_SPI_DW_MMIO=m
CONFIG_SPI_DLN2=m
CONFIG_SPI_GPIO=m
# CONFIG_SPI_INTEL_PCI is not set
# CONFIG_SPI_INTEL_PLATFORM is not set
CONFIG_SPI_LM70_LLP=m
# CONFIG_SPI_MICROCHIP_CORE is not set
# CONFIG_SPI_MICROCHIP_CORE_QSPI is not set
CONFIG_SPI_LANTIQ_SSC=m
CONFIG_SPI_OC_TINY=m
# CONFIG_SPI_PCI1XXXX is not set
CONFIG_SPI_PXA2XX=m
CONFIG_SPI_PXA2XX_PCI=m
CONFIG_SPI_SC18IS602=m
CONFIG_SPI_SIFIVE=m
CONFIG_SPI_MXIC=m
CONFIG_SPI_XCOMM=m
# CONFIG_SPI_XILINX is not set
CONFIG_SPI_ZYNQMP_GQSPI=m
CONFIG_SPI_AMD=m

#
# SPI Multiplexer support
#
CONFIG_SPI_MUX=m

#
# SPI Protocol Masters
#
CONFIG_SPI_SPIDEV=m
CONFIG_SPI_LOOPBACK_TEST=m
CONFIG_SPI_TLE62X0=m
CONFIG_SPI_SLAVE=y
CONFIG_SPI_SLAVE_TIME=m
CONFIG_SPI_SLAVE_SYSTEM_CONTROL=m
CONFIG_SPI_DYNAMIC=y
CONFIG_SPMI=m
CONFIG_SPMI_HISI3670=m
CONFIG_HSI=m
CONFIG_HSI_BOARDINFO=y

#
# HSI controllers
#

#
# HSI clients
#
CONFIG_HSI_CHAR=m
CONFIG_PPS=y
# CONFIG_PPS_DEBUG is not set

#
# PPS clients support
#
# CONFIG_PPS_CLIENT_KTIMER is not set
CONFIG_PPS_CLIENT_LDISC=m
CONFIG_PPS_CLIENT_PARPORT=m
CONFIG_PPS_CLIENT_GPIO=m

#
# PPS generators support
#

#
# PTP clock support
#
CONFIG_PTP_1588_CLOCK=y
CONFIG_PTP_1588_CLOCK_OPTIONAL=y
CONFIG_DP83640_PHY=m
CONFIG_PTP_1588_CLOCK_INES=m
CONFIG_PTP_1588_CLOCK_KVM=m
CONFIG_PTP_1588_CLOCK_IDT82P33=m
CONFIG_PTP_1588_CLOCK_IDTCM=m
# CONFIG_PTP_1588_CLOCK_FC3W is not set
# CONFIG_PTP_1588_CLOCK_MOCK is not set
CONFIG_PTP_1588_CLOCK_VMW=m
CONFIG_PTP_1588_CLOCK_OCP=m
# CONFIG_PTP_DFL_TOD is not set

# end of PTP clock support

CONFIG_PINCTRL=y
CONFIG_PINMUX=y
CONFIG_PINCONF=y
CONFIG_GENERIC_PINCONF=y
# CONFIG_DEBUG_PINCTRL is not set
CONFIG_PINCTRL_AMD=y
# CONFIG_PINCTRL_CY8C95X0 is not set
CONFIG_PINCTRL_DA9062=m
CONFIG_PINCTRL_MCP23S08_I2C=m
CONFIG_PINCTRL_MCP23S08_SPI=m
CONFIG_PINCTRL_MCP23S08=m
CONFIG_PINCTRL_SX150X=y
# CONFIG_PINCTRL_CS42L43 is not set
CONFIG_PINCTRL_MADERA=m
CONFIG_PINCTRL_CS47L15=y
CONFIG_PINCTRL_CS47L35=y
CONFIG_PINCTRL_CS47L85=y
CONFIG_PINCTRL_CS47L90=y
CONFIG_PINCTRL_CS47L92=y

#
# Intel pinctrl drivers
#
CONFIG_PINCTRL_BAYTRAIL=y
CONFIG_PINCTRL_CHERRYVIEW=y
CONFIG_PINCTRL_LYNXPOINT=m
CONFIG_PINCTRL_INTEL=y
# CONFIG_PINCTRL_INTEL_PLATFORM is not set
CONFIG_PINCTRL_ALDERLAKE=m
CONFIG_PINCTRL_BROXTON=m
CONFIG_PINCTRL_CANNONLAKE=m
CONFIG_PINCTRL_CEDARFORK=m
CONFIG_PINCTRL_DENVERTON=m
CONFIG_PINCTRL_ELKHARTLAKE=m
CONFIG_PINCTRL_EMMITSBURG=m
CONFIG_PINCTRL_GEMINILAKE=m
CONFIG_PINCTRL_ICELAKE=m
CONFIG_PINCTRL_JASPERLAKE=m
CONFIG_PINCTRL_LAKEFIELD=m
CONFIG_PINCTRL_LEWISBURG=m
# CONFIG_PINCTRL_METEORLAKE is not set
# CONFIG_PINCTRL_METEORPOINT is not set
CONFIG_PINCTRL_SUNRISEPOINT=m
CONFIG_PINCTRL_TIGERLAKE=m

# end of Intel pinctrl drivers

#
# Renesas pinctrl drivers
#

# end of Renesas pinctrl drivers

CONFIG_GPIOLIB=y
CONFIG_GPIOLIB_FASTPATH_LIMIT=512
CONFIG_GPIO_ACPI=y
CONFIG_GPIOLIB_IRQCHIP=y
# CONFIG_DEBUG_GPIO is not set
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_CDEV=y
# CONFIG_GPIO_CDEV_V1 is not set
CONFIG_GPIO_GENERIC=m
CONFIG_GPIO_REGMAP=m
CONFIG_GPIO_MAX730X=m
CONFIG_GPIO_IDIO_16=m

#
# Memory mapped GPIO drivers
#
CONFIG_GPIO_AMDPT=m
CONFIG_GPIO_DWAPB=m
CONFIG_GPIO_EXAR=m
CONFIG_GPIO_GENERIC_PLATFORM=m
CONFIG_GPIO_ICH=m
CONFIG_GPIO_MB86S7X=m
CONFIG_GPIO_MENZ127=m
CONFIG_GPIO_SIOX=m
CONFIG_GPIO_AMD_FCH=m

# end of Memory mapped GPIO drivers

#
# Port-mapped I/O GPIO drivers
#
CONFIG_GPIO_VX855=m
CONFIG_GPIO_I8255=m
CONFIG_GPIO_104_DIO_48E=m
CONFIG_GPIO_104_IDIO_16=m
CONFIG_GPIO_104_IDI_48=m
CONFIG_GPIO_F7188X=m
CONFIG_GPIO_GPIO_MM=m
CONFIG_GPIO_IT87=m
CONFIG_GPIO_SCH=m
CONFIG_GPIO_SCH311X=m
CONFIG_GPIO_WINBOND=m
CONFIG_GPIO_WS16C48=m

# end of Port-mapped I/O GPIO drivers

#
# I2C GPIO expanders
#
# CONFIG_GPIO_FXL6408 is not set
# CONFIG_GPIO_DS4520 is not set
CONFIG_GPIO_MAX7300=m
CONFIG_GPIO_MAX732X=m
CONFIG_GPIO_PCA953X=m
CONFIG_GPIO_PCA953X_IRQ=y
CONFIG_GPIO_PCA9570=m
CONFIG_GPIO_PCF857X=m
CONFIG_GPIO_TPIC2810=m

# end of I2C GPIO expanders

#
# MFD GPIO expanders
#
CONFIG_GPIO_ADP5520=m
CONFIG_GPIO_ARIZONA=m
CONFIG_GPIO_BD9571MWV=m
# CONFIG_GPIO_CROS_EC is not set
CONFIG_GPIO_CRYSTAL_COVE=y
CONFIG_GPIO_DA9052=m
CONFIG_GPIO_DA9055=m
CONFIG_GPIO_DLN2=m
# CONFIG_GPIO_ELKHARTLAKE is not set
CONFIG_GPIO_JANZ_TTL=m
CONFIG_GPIO_KEMPLD=m
CONFIG_GPIO_LP3943=m
CONFIG_GPIO_LP873X=m
CONFIG_GPIO_MADERA=m
CONFIG_GPIO_PALMAS=y
CONFIG_GPIO_RC5T583=y
CONFIG_GPIO_TPS65086=m
CONFIG_GPIO_TPS6586X=y
CONFIG_GPIO_TPS65910=y
CONFIG_GPIO_TPS65912=m
CONFIG_GPIO_TPS68470=m
CONFIG_GPIO_TQMX86=m
CONFIG_GPIO_TWL4030=m
CONFIG_GPIO_TWL6040=m
CONFIG_GPIO_WHISKEY_COVE=m
CONFIG_GPIO_WM831X=m
CONFIG_GPIO_WM8350=m
CONFIG_GPIO_WM8994=m

# end of MFD GPIO expanders

#
# PCI GPIO expanders
#
CONFIG_GPIO_AMD8111=m
CONFIG_GPIO_ML_IOH=m
CONFIG_GPIO_PCI_IDIO_16=m
CONFIG_GPIO_PCIE_IDIO_24=m
CONFIG_GPIO_RDC321X=m

# end of PCI GPIO expanders

#
# SPI GPIO expanders
#
CONFIG_GPIO_MAX3191X=m
CONFIG_GPIO_MAX7301=m
CONFIG_GPIO_MC33880=m
CONFIG_GPIO_PISOSR=m
CONFIG_GPIO_XRA1403=m

# end of SPI GPIO expanders

#
# USB GPIO expanders
#
CONFIG_GPIO_VIPERBOARD=m

# end of USB GPIO expanders

#
# Virtual GPIO drivers
#
CONFIG_GPIO_AGGREGATOR=m
# CONFIG_GPIO_LATCH is not set
# CONFIG_GPIO_MOCKUP is not set
CONFIG_GPIO_VIRTIO=m
# CONFIG_GPIO_SIM is not set

# end of Virtual GPIO drivers

CONFIG_W1=m
CONFIG_W1_CON=y

#
# 1-wire Bus Masters
#
# CONFIG_W1_MASTER_AMD_AXI is not set
CONFIG_W1_MASTER_MATROX=m
CONFIG_W1_MASTER_DS2490=m
CONFIG_W1_MASTER_DS2482=m
CONFIG_W1_MASTER_GPIO=m
CONFIG_W1_MASTER_SGI=m
# CONFIG_W1_MASTER_UART is not set

# end of 1-wire Bus Masters

#
# 1-wire Slaves
#
CONFIG_W1_SLAVE_THERM=m
CONFIG_W1_SLAVE_SMEM=m
CONFIG_W1_SLAVE_DS2405=m
CONFIG_W1_SLAVE_DS2408=m
CONFIG_W1_SLAVE_DS2408_READBACK=y
CONFIG_W1_SLAVE_DS2413=m
CONFIG_W1_SLAVE_DS2406=m
CONFIG_W1_SLAVE_DS2423=m
CONFIG_W1_SLAVE_DS2805=m
CONFIG_W1_SLAVE_DS2430=m
CONFIG_W1_SLAVE_DS2431=m
CONFIG_W1_SLAVE_DS2433=m
# CONFIG_W1_SLAVE_DS2433_CRC is not set
CONFIG_W1_SLAVE_DS2438=m
CONFIG_W1_SLAVE_DS250X=m
CONFIG_W1_SLAVE_DS2780=m
CONFIG_W1_SLAVE_DS2781=m
CONFIG_W1_SLAVE_DS28E04=m
CONFIG_W1_SLAVE_DS28E17=m

# end of 1-wire Slaves

CONFIG_POWER_RESET=y CONFIG_POWER_RESET_ATC260X=m CONFIG_POWER_RESET_MT6323=y CONFIG_POWER_RESET_RESTART=y CONFIG_POWER_RESET_TPS65086=y CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set CONFIG_POWER_SUPPLY_HWMON=y CONFIG_GENERIC_ADC_BATTERY=m
# CONFIG_IP5XXX_POWER is not set CONFIG_MAX8925_POWER=m CONFIG_WM831X_BACKUP=m CONFIG_WM831X_POWER=m CONFIG_WM8350_POWER=m
CONFIG_TEST_POWER=m CONFIG_BATTERY_88PM860X=m CONFIG_CHARGER_ADP5061=m CONFIG_BATTERY_CW2015=m CONFIG_BATTERY_DS2760=m CONFIG_BATTERY_DS2780=m CONFIG_BATTERY_DS2781=m CONFIG_BATTERY_DS2782=m
# CONFIG_BATTERY_SAMSUNG_SDI is not set CONFIG_BATTERY_SBS=m
CONFIG_CHARGER_SBS=m
CONFIG_MANAGER_SBS=m CONFIG_BATTERY_BQ27XXX=m CONFIG_BATTERY_BQ27XXX_I2C=m CONFIG_BATTERY_BQ27XXX_HDQ=m
# CONFIG_BATTERY_BQ27XXX_DT_UPDATES_NVM is not set CONFIG_BATTERY_DA9030=m CONFIG_BATTERY_DA9052=m CONFIG_CHARGER_DA9150=m CONFIG_BATTERY_DA9150=m CONFIG_CHARGER_AXP20X=m CONFIG_BATTERY_AXP20X=m CONFIG_AXP20X_POWER=m CONFIG_AXP288_CHARGER=m CONFIG_AXP288_FUEL_GAUGE=m CONFIG_BATTERY_MAX17040=m CONFIG_BATTERY_MAX17042=m CONFIG_BATTERY_MAX1721X=m CONFIG_BATTERY_TWL4030_MADC=m CONFIG_CHARGER_88PM860X=m CONFIG_CHARGER_PCF50633=m CONFIG_BATTERY_RX51=m CONFIG_CHARGER_ISP1704=m CONFIG_CHARGER_MAX8903=m CONFIG_CHARGER_TWL4030=m CONFIG_CHARGER_LP8727=m CONFIG_CHARGER_LP8788=m CONFIG_CHARGER_GPIO=m CONFIG_CHARGER_MANAGER=y CONFIG_CHARGER_LT3651=m CONFIG_CHARGER_LTC4162L=m CONFIG_CHARGER_MAX14577=m CONFIG_CHARGER_MAX77693=m
# CONFIG_CHARGER_MAX77976 is not set CONFIG_CHARGER_MAX8997=m CONFIG_CHARGER_MAX8998=m CONFIG_CHARGER_MP2629=m CONFIG_CHARGER_MT6360=m CONFIG_CHARGER_BQ2415X=m CONFIG_CHARGER_BQ24190=m CONFIG_CHARGER_BQ24257=m CONFIG_CHARGER_BQ24735=m CONFIG_CHARGER_BQ2515X=m CONFIG_CHARGER_BQ25890=m CONFIG_CHARGER_BQ25980=m CONFIG_CHARGER_BQ256XX=m CONFIG_CHARGER_SMB347=m CONFIG_CHARGER_TPS65090=m CONFIG_BATTERY_GAUGE_LTC2941=m CONFIG_BATTERY_GOLDFISH=m CONFIG_BATTERY_RT5033=m
# CONFIG_CHARGER_RT5033 is not set CONFIG_CHARGER_RT9455=m
# CONFIG_CHARGER_RT9467 is not set
# CONFIG_CHARGER_RT9471 is not set CONFIG_CHARGER_CROS_USBPD=m CONFIG_CHARGER_CROS_PCHG=m CONFIG_CHARGER_BD99954=m CONFIG_CHARGER_WILCO=m CONFIG_BATTERY_SURFACE=m CONFIG_CHARGER_SURFACE=m
# CONFIG_BATTERY_UG3105 is not set
# CONFIG_FUEL_GAUGE_MM8013 is not set CONFIG_HWMON=y
CONFIG_HWMON_VID=m
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Native drivers
#
CONFIG_SENSORS_ABITUGURU=m
CONFIG_SENSORS_ABITUGURU3=m
CONFIG_SENSORS_AD7314=m
CONFIG_SENSORS_AD7414=m
CONFIG_SENSORS_AD7418=m
CONFIG_SENSORS_ADM1025=m
CONFIG_SENSORS_ADM1026=m
CONFIG_SENSORS_ADM1029=m
CONFIG_SENSORS_ADM1031=m
CONFIG_SENSORS_ADM1177=m
CONFIG_SENSORS_ADM9240=m
CONFIG_SENSORS_ADT7X10=m
CONFIG_SENSORS_ADT7310=m
CONFIG_SENSORS_ADT7410=m
CONFIG_SENSORS_ADT7411=m
CONFIG_SENSORS_ADT7462=m
CONFIG_SENSORS_ADT7470=m
CONFIG_SENSORS_ADT7475=m
CONFIG_SENSORS_AHT10=m
CONFIG_SENSORS_AQUACOMPUTER_D5NEXT=m
CONFIG_SENSORS_AS370=m
CONFIG_SENSORS_ASC7621=m
# CONFIG_SENSORS_ASUS_ROG_RYUJIN is not set
CONFIG_SENSORS_AXI_FAN_CONTROL=m
CONFIG_SENSORS_K8TEMP=m
CONFIG_SENSORS_K10TEMP=m
CONFIG_SENSORS_FAM15H_POWER=m
CONFIG_SENSORS_APPLESMC=m
CONFIG_SENSORS_ASB100=m
CONFIG_SENSORS_ATXP1=m
# CONFIG_SENSORS_CHIPCAP2 is not set
CONFIG_SENSORS_CORSAIR_CPRO=m
CONFIG_SENSORS_CORSAIR_PSU=m
CONFIG_SENSORS_DRIVETEMP=m
CONFIG_SENSORS_DS620=m
CONFIG_SENSORS_DS1621=m
CONFIG_SENSORS_DELL_SMM=m
# CONFIG_I8K is not set
CONFIG_SENSORS_DA9052_ADC=m
CONFIG_SENSORS_DA9055=m
CONFIG_SENSORS_I5K_AMB=m
CONFIG_SENSORS_F71805F=m
CONFIG_SENSORS_F71882FG=m
CONFIG_SENSORS_F75375S=m
CONFIG_SENSORS_MC13783_ADC=m
CONFIG_SENSORS_FSCHMD=m
CONFIG_SENSORS_FTSTEUTATES=m
# CONFIG_SENSORS_GIGABYTE_WATERFORCE is not set
CONFIG_SENSORS_GL518SM=m
CONFIG_SENSORS_GL520SM=m
CONFIG_SENSORS_G760A=m
CONFIG_SENSORS_G762=m
CONFIG_SENSORS_HIH6130=m
# CONFIG_SENSORS_HS3001 is not set
CONFIG_SENSORS_IBMAEM=m
CONFIG_SENSORS_IBMPEX=m
CONFIG_SENSORS_IIO_HWMON=m
CONFIG_SENSORS_I5500=m
CONFIG_SENSORS_CORETEMP=m
CONFIG_SENSORS_IT87=m
CONFIG_SENSORS_JC42=m
# CONFIG_SENSORS_POWERZ is not set
CONFIG_SENSORS_POWR1220=m
CONFIG_SENSORS_LINEAGE=m
CONFIG_SENSORS_LTC2945=m
CONFIG_SENSORS_LTC2947=m
CONFIG_SENSORS_LTC2947_I2C=m
CONFIG_SENSORS_LTC2947_SPI=m
CONFIG_SENSORS_LTC2990=m
# CONFIG_SENSORS_LTC2991 is not set
CONFIG_SENSORS_LTC2992=m
CONFIG_SENSORS_LTC4151=m
CONFIG_SENSORS_LTC4215=m
CONFIG_SENSORS_LTC4222=m
CONFIG_SENSORS_LTC4245=m
CONFIG_SENSORS_LTC4260=m
CONFIG_SENSORS_LTC4261=m
# CONFIG_SENSORS_LTC4282 is not set
CONFIG_SENSORS_MAX1111=m
CONFIG_SENSORS_MAX127=m
CONFIG_SENSORS_MAX16065=m
CONFIG_SENSORS_MAX1619=m
CONFIG_SENSORS_MAX1668=m
CONFIG_SENSORS_MAX197=m
CONFIG_SENSORS_MAX31722=m
CONFIG_SENSORS_MAX31730=m
# CONFIG_SENSORS_MAX31760 is not set
# CONFIG_MAX31827 is not set
# CONFIG_SENSORS_MAX6620 is not set
CONFIG_SENSORS_MAX6621=m
CONFIG_SENSORS_MAX6639=m
CONFIG_SENSORS_MAX6650=m
CONFIG_SENSORS_MAX6697=m
CONFIG_SENSORS_MAX31790=m
# CONFIG_SENSORS_MC34VR500 is not set
CONFIG_SENSORS_MCP3021=m
CONFIG_SENSORS_MLXREG_FAN=m
CONFIG_SENSORS_TC654=m
CONFIG_SENSORS_TPS23861=m
CONFIG_SENSORS_MENF21BMC_HWMON=m
CONFIG_SENSORS_MR75203=m
CONFIG_SENSORS_ADCXX=m
CONFIG_SENSORS_LM63=m
CONFIG_SENSORS_LM70=m
CONFIG_SENSORS_LM73=m
CONFIG_SENSORS_LM75=m
CONFIG_SENSORS_LM77=m
CONFIG_SENSORS_LM78=m
CONFIG_SENSORS_LM80=m
CONFIG_SENSORS_LM83=m
CONFIG_SENSORS_LM85=m
CONFIG_SENSORS_LM87=m
CONFIG_SENSORS_LM90=m
CONFIG_SENSORS_LM92=m
CONFIG_SENSORS_LM93=m
CONFIG_SENSORS_LM95234=m
CONFIG_SENSORS_LM95241=m
CONFIG_SENSORS_LM95245=m
CONFIG_SENSORS_PC87360=m
CONFIG_SENSORS_PC87427=m
CONFIG_SENSORS_NTC_THERMISTOR=m
CONFIG_SENSORS_NCT6683=m
CONFIG_SENSORS_NCT6775_CORE=m
CONFIG_SENSORS_NCT6775=m
# CONFIG_SENSORS_NCT6775_I2C is not set
CONFIG_SENSORS_NCT7802=m
CONFIG_SENSORS_NCT7904=m
CONFIG_SENSORS_NPCM7XX=m
CONFIG_SENSORS_NZXT_KRAKEN2=m
# CONFIG_SENSORS_NZXT_KRAKEN3 is not set
# CONFIG_SENSORS_NZXT_SMART2 is not set
# CONFIG_SENSORS_OCC_P8_I2C is not set
# CONFIG_SENSORS_OXP is not set
CONFIG_SENSORS_PCF8591=m
CONFIG_PMBUS=m
CONFIG_SENSORS_PMBUS=m
# CONFIG_SENSORS_ACBEL_FSG032 is not set
CONFIG_SENSORS_ADM1266=m
CONFIG_SENSORS_ADM1275=m
CONFIG_SENSORS_BEL_PFE=m
CONFIG_SENSORS_BPA_RS600=m
# CONFIG_SENSORS_DELTA_AHE50DC_FAN is not set
CONFIG_SENSORS_FSP_3Y=m
CONFIG_SENSORS_IBM_CFFPS=m
CONFIG_SENSORS_DPS920AB=m
CONFIG_SENSORS_INSPUR_IPSPS=m
CONFIG_SENSORS_IR35221=m
CONFIG_SENSORS_IR36021=m
CONFIG_SENSORS_IR38064=m
# CONFIG_SENSORS_IR38064_REGULATOR is not set
CONFIG_SENSORS_IRPS5401=m
CONFIG_SENSORS_ISL68137=m
CONFIG_SENSORS_LM25066=m
# CONFIG_SENSORS_LM25066_REGULATOR is not set
# CONFIG_SENSORS_LT7182S is not set
CONFIG_SENSORS_LTC2978=m
CONFIG_SENSORS_LTC2978_REGULATOR=y
CONFIG_SENSORS_LTC3815=m
# CONFIG_SENSORS_LTC4286 is not set
CONFIG_SENSORS_MAX15301=m
CONFIG_SENSORS_MAX16064=m
CONFIG_SENSORS_MAX16601=m
CONFIG_SENSORS_MAX20730=m
CONFIG_SENSORS_MAX20751=m
CONFIG_SENSORS_MAX31785=m
CONFIG_SENSORS_MAX34440=m
CONFIG_SENSORS_MAX8688=m
# CONFIG_SENSORS_MP2856 is not set
CONFIG_SENSORS_MP2888=m
CONFIG_SENSORS_MP2975=m
# CONFIG_SENSORS_MP2975_REGULATOR is not set
# CONFIG_SENSORS_MP5023 is not set
# CONFIG_SENSORS_MP5990 is not set
# CONFIG_SENSORS_MPQ7932 is not set
# CONFIG_SENSORS_MPQ8785 is not set
CONFIG_SENSORS_PIM4328=m
# CONFIG_SENSORS_PLI1209BC is not set
CONFIG_SENSORS_PM6764TR=m
CONFIG_SENSORS_PXE1610=m
CONFIG_SENSORS_Q54SJ108A2=m
CONFIG_SENSORS_STPDDC60=m
# CONFIG_SENSORS_TDA38640 is not set
CONFIG_SENSORS_TPS40422=m
CONFIG_SENSORS_TPS53679=m
# CONFIG_SENSORS_TPS546D24 is not set
CONFIG_SENSORS_UCD9000=m
CONFIG_SENSORS_UCD9200=m
# CONFIG_SENSORS_XDPE152 is not set
CONFIG_SENSORS_XDPE122=m
# CONFIG_SENSORS_XDPE122_REGULATOR is not set
CONFIG_SENSORS_ZL6100=m
# CONFIG_SENSORS_PT5161L is not set
CONFIG_SENSORS_SBTSI=m
CONFIG_SENSORS_SBRMI=m
CONFIG_SENSORS_SHT15=m
CONFIG_SENSORS_SHT21=m
CONFIG_SENSORS_SHT3x=m
CONFIG_SENSORS_SHT4x=m
CONFIG_SENSORS_SHTC1=m
CONFIG_SENSORS_SIS5595=m
CONFIG_SENSORS_DME1737=m
CONFIG_SENSORS_EMC1403=m
CONFIG_SENSORS_EMC2103=m
# CONFIG_SENSORS_EMC2305 is not set
CONFIG_SENSORS_EMC6W201=m
CONFIG_SENSORS_SMSC47M1=m
CONFIG_SENSORS_SMSC47M192=m
CONFIG_SENSORS_SMSC47B397=m
CONFIG_SENSORS_SCH56XX_COMMON=m
CONFIG_SENSORS_SCH5627=m
CONFIG_SENSORS_SCH5636=m
CONFIG_SENSORS_STTS751=m
# CONFIG_SENSORS_SURFACE_FAN is not set
CONFIG_SENSORS_ADC128D818=m
CONFIG_SENSORS_ADS7828=m
CONFIG_SENSORS_ADS7871=m
CONFIG_SENSORS_AMC6821=m
CONFIG_SENSORS_INA209=m
CONFIG_SENSORS_INA2XX=m
# CONFIG_SENSORS_INA238 is not set
CONFIG_SENSORS_INA3221=m
CONFIG_SENSORS_TC74=m
CONFIG_SENSORS_THMC50=m
CONFIG_SENSORS_TMP102=m
CONFIG_SENSORS_TMP103=m
CONFIG_SENSORS_TMP108=m
CONFIG_SENSORS_TMP401=m
CONFIG_SENSORS_TMP421=m
# CONFIG_SENSORS_TMP464 is not set
CONFIG_SENSORS_TMP513=m
CONFIG_SENSORS_VIA_CPUTEMP=m
CONFIG_SENSORS_VIA686A=m
CONFIG_SENSORS_VT1211=m
CONFIG_SENSORS_VT8231=m
CONFIG_SENSORS_W83773G=m
CONFIG_SENSORS_W83781D=m
CONFIG_SENSORS_W83791D=m
CONFIG_SENSORS_W83792D=m
CONFIG_SENSORS_W83793=m
CONFIG_SENSORS_W83795=m
# CONFIG_SENSORS_W83795_FANCTRL is not set
CONFIG_SENSORS_W83L785TS=m
CONFIG_SENSORS_W83L786NG=m
CONFIG_SENSORS_W83627HF=m
CONFIG_SENSORS_W83627EHF=m
CONFIG_SENSORS_WM831X=m
CONFIG_SENSORS_WM8350=m
CONFIG_SENSORS_XGENE=m

#
# ACPI drivers
#
CONFIG_SENSORS_ACPI_POWER=m CONFIG_SENSORS_ATK0110=m
# CONFIG_SENSORS_ASUS_WMI is not set
# CONFIG_SENSORS_ASUS_EC is not set
# CONFIG_SENSORS_HP_WMI is not set CONFIG_THERMAL=y
CONFIG_THERMAL_NETLINK=y CONFIG_THERMAL_STATISTICS=y
# CONFIG_THERMAL_DEBUGFS is not set CONFIG_THERMAL_EMERGENCY_POWEROFF_DELAY_MS=0 CONFIG_THERMAL_HWMON=y CONFIG_THERMAL_DEFAULT_GOV_STEP_WISE=y
# CONFIG_THERMAL_DEFAULT_GOV_FAIR_SHARE is not set # CONFIG_THERMAL_DEFAULT_GOV_USER_SPACE is not set # CONFIG_THERMAL_DEFAULT_GOV_POWER_ALLOCATOR is not set # CONFIG_THERMAL_DEFAULT_GOV_BANG_BANG is not set CONFIG_THERMAL_GOV_FAIR_SHARE=y CONFIG_THERMAL_GOV_STEP_WISE=y CONFIG_THERMAL_GOV_BANG_BANG=y CONFIG_THERMAL_GOV_USER_SPACE=y CONFIG_THERMAL_GOV_POWER_ALLOCATOR=y CONFIG_DEVFREQ_THERMAL=y CONFIG_THERMAL_EMULATION=y

#
# Intel thermal drivers
#
CONFIG_INTEL_POWERCLAMP=m
CONFIG_X86_THERMAL_VECTOR=y
CONFIG_INTEL_TCC=y
CONFIG_X86_PKG_TEMP_THERMAL=m
CONFIG_INTEL_SOC_DTS_IOSF_CORE=m
CONFIG_INTEL_SOC_DTS_THERMAL=m

#
# ACPI INT340X thermal drivers
#
CONFIG_INT340X_THERMAL=m
CONFIG_ACPI_THERMAL_REL=m
CONFIG_INT3406_THERMAL=m
CONFIG_PROC_THERMAL_MMIO_RAPL=m

# end of ACPI INT340X thermal drivers

CONFIG_INTEL_BXT_PMIC_THERMAL=m
CONFIG_INTEL_PCH_THERMAL=m
CONFIG_INTEL_TCC_COOLING=m
# CONFIG_INTEL_HFI_THERMAL is not set

# end of Intel thermal drivers

CONFIG_GENERIC_ADC_THERMAL=m
CONFIG_WATCHDOG=y
CONFIG_WATCHDOG_CORE=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
CONFIG_WATCHDOG_HANDLE_BOOT_ENABLED=y
CONFIG_WATCHDOG_OPEN_TIMEOUT=0
CONFIG_WATCHDOG_SYSFS=y
# CONFIG_WATCHDOG_HRTIMER_PRETIMEOUT is not set

#
# Watchdog Pretimeout Governors
#
# CONFIG_WATCHDOG_PRETIMEOUT_GOV is not set

#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=m
# CONFIG_CROS_EC_WATCHDOG is not set
CONFIG_DA9052_WATCHDOG=m
CONFIG_DA9055_WATCHDOG=m
CONFIG_DA9063_WATCHDOG=m
CONFIG_DA9062_WATCHDOG=m
CONFIG_MENF21BMC_WATCHDOG=m
CONFIG_MENZ069_WATCHDOG=m
CONFIG_WDAT_WDT=m
CONFIG_WM831X_WATCHDOG=m
CONFIG_WM8350_WATCHDOG=m
CONFIG_XILINX_WATCHDOG=m
CONFIG_ZIIRAVE_WATCHDOG=m
CONFIG_RAVE_SP_WATCHDOG=m
CONFIG_MLX_WDT=m
CONFIG_CADENCE_WATCHDOG=m
CONFIG_DW_WATCHDOG=m
CONFIG_TWL4030_WATCHDOG=m
CONFIG_MAX63XX_WATCHDOG=m
CONFIG_RETU_WATCHDOG=m
CONFIG_ACQUIRE_WDT=m
CONFIG_ADVANTECH_WDT=m
# CONFIG_ADVANTECH_EC_WDT is not set
CONFIG_ALIM1535_WDT=m
CONFIG_ALIM7101_WDT=m
CONFIG_EBC_C384_WDT=m
# CONFIG_EXAR_WDT is not set
CONFIG_F71808E_WDT=m
CONFIG_SP5100_TCO=y
CONFIG_SBC_FITPC2_WATCHDOG=m
CONFIG_EUROTECH_WDT=m
CONFIG_IB700_WDT=m
CONFIG_IBMASR=m
CONFIG_WAFER_WDT=m
CONFIG_I6300ESB_WDT=m
CONFIG_IE6XX_WDT=m
CONFIG_ITCO_WDT=m
CONFIG_ITCO_VENDOR_SUPPORT=y
CONFIG_IT8712F_WDT=m
CONFIG_IT87_WDT=m
CONFIG_HP_WATCHDOG=m
CONFIG_HPWDT_NMI_DECODING=y
CONFIG_KEMPLD_WDT=m
CONFIG_SC1200_WDT=m
CONFIG_PC87413_WDT=m
CONFIG_NV_TCO=m
CONFIG_60XX_WDT=m
CONFIG_CPU5_WDT=m
CONFIG_SMSC_SCH311X_WDT=m
CONFIG_SMSC37B787_WDT=m
CONFIG_TQMX86_WDT=m
CONFIG_VIA_WDT=m
CONFIG_W83627HF_WDT=m
CONFIG_W83877F_WDT=m
CONFIG_W83977F_WDT=m
CONFIG_MACHZ_WDT=m
CONFIG_SBC_EPX_C3_WATCHDOG=m
CONFIG_INTEL_MEI_WDT=m
CONFIG_NI903X_WDT=m
CONFIG_NIC7018_WDT=m
CONFIG_MEN_A21_WDT=m
CONFIG_XEN_WDT=m

#
# PCI-based Watchdog Cards
#
CONFIG_PCIPCWATCHDOG=m
CONFIG_WDTPCI=m

#
# USB-based Watchdog Cards
#
CONFIG_USBPCWATCHDOG=m
CONFIG_SSB_POSSIBLE=y
CONFIG_SSB=m
CONFIG_SSB_SPROM=y
CONFIG_SSB_BLOCKIO=y
CONFIG_SSB_PCIHOST_POSSIBLE=y
CONFIG_SSB_PCIHOST=y
CONFIG_SSB_B43_PCI_BRIDGE=y
CONFIG_SSB_PCMCIAHOST_POSSIBLE=y
# CONFIG_SSB_PCMCIAHOST is not set
CONFIG_SSB_SDIOHOST_POSSIBLE=y
CONFIG_SSB_SDIOHOST=y
CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
CONFIG_SSB_DRIVER_PCICORE=y
CONFIG_SSB_DRIVER_GPIO=y
CONFIG_BCMA_POSSIBLE=y
CONFIG_BCMA=m
CONFIG_BCMA_BLOCKIO=y
CONFIG_BCMA_HOST_PCI_POSSIBLE=y
CONFIG_BCMA_HOST_PCI=y
CONFIG_BCMA_HOST_SOC=y
CONFIG_BCMA_DRIVER_PCI=y
CONFIG_BCMA_SFLASH=y
CONFIG_BCMA_DRIVER_GMAC_CMN=y
CONFIG_BCMA_DRIVER_GPIO=y
# CONFIG_BCMA_DEBUG is not set

#
# Multifunction device drivers
#
CONFIG_MFD_CORE=y
CONFIG_MFD_AS3711=y
# CONFIG_MFD_SMPRO is not set
CONFIG_PMIC_ADP5520=y
CONFIG_MFD_AAT2870_CORE=y
CONFIG_MFD_BCM590XX=m
CONFIG_MFD_BD9571MWV=m
CONFIG_MFD_AXP20X=m
CONFIG_MFD_AXP20X_I2C=m
CONFIG_MFD_CROS_EC_DEV=m
CONFIG_MFD_CS42L43=m
# CONFIG_MFD_CS42L43_I2C is not set
CONFIG_MFD_CS42L43_SDW=m
CONFIG_MFD_MADERA=m
CONFIG_MFD_MADERA_I2C=m
CONFIG_MFD_MADERA_SPI=m
CONFIG_MFD_CS47L15=y
CONFIG_MFD_CS47L35=y
CONFIG_MFD_CS47L85=y
CONFIG_MFD_CS47L90=y
CONFIG_MFD_CS47L92=y
CONFIG_PMIC_DA903X=y
CONFIG_PMIC_DA9052=y
CONFIG_MFD_DA9052_SPI=y
CONFIG_MFD_DA9052_I2C=y
CONFIG_MFD_DA9055=y
CONFIG_MFD_DA9062=m
CONFIG_MFD_DA9063=y
CONFIG_MFD_DA9150=m
CONFIG_MFD_DLN2=m
CONFIG_MFD_MC13XXX=m
CONFIG_MFD_MC13XXX_SPI=m
CONFIG_MFD_MC13XXX_I2C=m
CONFIG_MFD_MP2629=m
CONFIG_MFD_INTEL_QUARK_I2C_GPIO=m
CONFIG_LPC_ICH=m
CONFIG_LPC_SCH=m
CONFIG_INTEL_SOC_PMIC=y
CONFIG_INTEL_SOC_PMIC_BXTWC=m
CONFIG_INTEL_SOC_PMIC_CHTWC=y
CONFIG_INTEL_SOC_PMIC_CHTDC_TI=m
CONFIG_INTEL_SOC_PMIC_MRFLD=m
CONFIG_MFD_INTEL_LPSS=m
CONFIG_MFD_INTEL_LPSS_ACPI=m
CONFIG_MFD_INTEL_LPSS_PCI=m
CONFIG_MFD_INTEL_PMC_BXT=m
CONFIG_MFD_IQS62X=m
CONFIG_MFD_JANZ_CMODIO=m
CONFIG_MFD_KEMPLD=m
CONFIG_MFD_88PM800=m
CONFIG_MFD_88PM805=m
CONFIG_MFD_88PM860X=y
CONFIG_MFD_MAX14577=y
# CONFIG_MFD_MAX77541 is not set
CONFIG_MFD_MAX77693=y
CONFIG_MFD_MAX77843=y
CONFIG_MFD_MAX8907=m
CONFIG_MFD_MAX8925=y
CONFIG_MFD_MAX8997=y
CONFIG_MFD_MAX8998=y
CONFIG_MFD_MT6360=m
# CONFIG_MFD_MT6370 is not set
CONFIG_MFD_MT6397=m
CONFIG_MFD_MENF21BMC=m
# CONFIG_MFD_OCELOT is not set
CONFIG_EZX_PCAP=y
CONFIG_MFD_VIPERBOARD=m
CONFIG_MFD_RETU=m
CONFIG_MFD_PCF50633=m
CONFIG_PCF50633_ADC=m
CONFIG_PCF50633_GPIO=m
# CONFIG_MFD_SY7636A is not set
CONFIG_MFD_RDC321X=m
CONFIG_MFD_RT4831=m
CONFIG_MFD_RT5033=m
# CONFIG_MFD_RT5120 is not set
CONFIG_MFD_RC5T583=y
CONFIG_MFD_SI476X_CORE=m
CONFIG_MFD_SM501=m
CONFIG_MFD_SM501_GPIO=y
CONFIG_MFD_SKY81452=m
CONFIG_MFD_SYSCON=y
CONFIG_MFD_LP3943=m
CONFIG_MFD_LP8788=y
CONFIG_MFD_TI_LMU=m
CONFIG_MFD_PALMAS=y
CONFIG_TPS6105X=m
CONFIG_TPS65010=m
CONFIG_TPS6507X=m
CONFIG_MFD_TPS65086=m
CONFIG_MFD_TPS65090=y
CONFIG_MFD_TI_LP873X=m
CONFIG_MFD_TPS6586X=y
CONFIG_MFD_TPS65910=y
CONFIG_MFD_TPS65912=y
CONFIG_MFD_TPS65912_I2C=y
CONFIG_MFD_TPS65912_SPI=y
# CONFIG_MFD_TPS6594_I2C is not set
# CONFIG_MFD_TPS6594_SPI is not set
CONFIG_TWL4030_CORE=y
CONFIG_MFD_TWL4030_AUDIO=y
CONFIG_TWL6040_CORE=y
CONFIG_MFD_WL1273_CORE=m
CONFIG_MFD_LM3533=m
CONFIG_MFD_TQMX86=m
CONFIG_MFD_VX855=m
CONFIG_MFD_ARIZONA=m
CONFIG_MFD_ARIZONA_I2C=m
CONFIG_MFD_ARIZONA_SPI=m
CONFIG_MFD_CS47L24=y
CONFIG_MFD_WM5102=y
CONFIG_MFD_WM5110=y
CONFIG_MFD_WM8997=y
CONFIG_MFD_WM8998=y
CONFIG_MFD_WM8400=y
CONFIG_MFD_WM831X=y
CONFIG_MFD_WM831X_I2C=y
CONFIG_MFD_WM831X_SPI=y
CONFIG_MFD_WM8350=y
CONFIG_MFD_WM8350_I2C=y
CONFIG_MFD_WM8994=m
CONFIG_MFD_WCD934X=m
CONFIG_MFD_ATC260X=m
CONFIG_MFD_ATC260X_I2C=m
CONFIG_RAVE_SP_CORE=m
# CONFIG_MFD_INTEL_M10_BMC_SPI is not set
# CONFIG_MFD_INTEL_M10_BMC_PMCI is not set

# end of Multifunction device drivers

5,983 lines not shown · raw message

Part of this message is not shown here. View the raw message.

#16 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 11 lines · K Prateek Nayak

I ran into a few issues when testing the series on top of tip:sched/core at commit 4475cd8bfd9b ("sched/balancing: Simplify the sg_status bitmask and use separate ->overloaded and ->overutilized flags"). All of these splats surfaced when running unixbench with Delayed Dequeue (echoing NO_DELAY_DEQUEUE to /sys/kernel/debug/sched/features seems to make the system stable when running Unixbench spawn)

Unixbench (https://github.com/kdlucas/byte-unixbench.git) command:

        ./Run spawn -c 512

That plus a hackbench loop works a treat.

Quoted 4 lines

Splats appear soon into the run. Following are the splats and their corresponding code blocks from my 3rd Generation EPYC system (2 x 64C/128T):

Seems a big box is not required. With a low fat sched config (no group sched), starting ./Run spawn -c 16 (cpus*2) along with a hackbench loop reliably blows my old i7-4790 box out of the water nearly instantly.

    DUMPFILE: vmcore
        CPUS: 8
        DATE: Mon Apr 15 07:20:29 CEST 2024
      UPTIME: 00:07:23
LOAD AVERAGE: 1632.20, 684.99, 291.84
       TASKS: 1401
    NODENAME: homer
     RELEASE: 6.9.0.g0bbac3f-master
     VERSION: #7 SMP Mon Apr 15 06:40:05 CEST 2024
     MACHINE: x86_64  (3591 Mhz)
      MEMORY: 16 GB
       PANIC: "Oops: 0000 [#1] SMP NOPTI" (check log for details)
         PID: 22664
     COMMAND: "hackbench"
        TASK: ffff888100acbf00  [THREAD_INFO: ffff888100acbf00]
         CPU: 5
       STATE: TASK_WAKING (PANIC)

crash> bt -sx
PID: 22664 TASK: ffff888100acbf00 CPU: 5 COMMAND: "hackbench"
#0 [ffff88817cc17920] machine_kexec+0x156 at ffffffff810642d6 #1 [ffff88817cc17970] __crash_kexec+0xd7 at ffffffff81153147 #2 [ffff88817cc17a28] crash_kexec+0x23 at ffffffff811535f3 #3 [ffff88817cc17a38] oops_end+0xbe at ffffffff810329be #4 [ffff88817cc17a58] page_fault_oops+0x81 at ffffffff81071951 #5 [ffff88817cc17ab8] exc_page_fault+0x62 at ffffffff8194f6f2 #6 [ffff88817cc17ae0] asm_exc_page_fault+0x22 at ffffffff81a00ba2

    [exception RIP: pick_task_fair+71]
    RIP: ffffffff810d5b57  RSP: ffff88817cc17b90  RFLAGS: 00010046
    RAX: 0000000000000000  RBX: ffff88840ed70ec0  RCX: 00000001d7ec138c
    RDX: ffffffffe7a7f400  RSI: 0000000000000000  RDI: 0000000000000000
    RBP: ffff88840ed70ec0   R8: 0000000000000c00   R9: 000000675402f79e
    R10: ffff88817cc17b30  R11: 00000000000000bb  R12: ffff88840ed70f40
    R13: ffffffff81f64f16  R14: ffff888100acc560  R15: ffff888100acbf00
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

#7 [ffff88817cc17bb0] pick_next_task_fair+0x42 at ffffffff810d92c2 #8 [ffff88817cc17be0] __schedule+0x10d at ffffffff8195936d
#9 [ffff88817cc17c50] schedule+0x1c at ffffffff81959ddc
#10 [ffff88817cc17c60] schedule_timeout+0x18c at ffffffff8195fc4c #11 [ffff88817cc17cc8] unix_stream_read_generic+0x2b7 at ffffffff81869917 #12 [ffff88817cc17da8] unix_stream_recvmsg+0x68 at ffffffff8186a2d8 #13 [ffff88817cc17de0] sock_read_iter+0x159 at ffffffff8170bd69 #14 [ffff88817cc17e70] vfs_read+0x2ce at ffffffff812f195e
#15 [ffff88817cc17ef8] ksys_read+0x40 at ffffffff812f21d0
#16 [ffff88817cc17f30] do_syscall_64+0x57 at ffffffff8194b947
#17 [ffff88817cc17f50] entry_SYSCALL_64_after_hwframe+0x76 at ffffffff81a0012b

    RIP: 00007f625660871e  RSP: 00007ffc75d48188  RFLAGS: 00000246
    RAX: ffffffffffffffda  RBX: 00007ffc75d48200  RCX: 00007f625660871e
    RDX: 0000000000000064  RSI: 00007ffc75d48190  RDI: 0000000000000007
    RBP: 00007ffc75d48260   R8: 00007ffc75d48140   R9: 00007f6256612010
    R10: 00007f62565f5070  R11: 0000000000000246  R12: 0000000000000064
    R13: 0000000000000000  R14: 0000000000000064  R15: 0000000000000000
    ORIG_RAX: 0000000000000000  CS: 0033  SS: 002b

crash> dis pick_task_fair+71
0xffffffff810d5b57 <pick_task_fair+71>: cmpb $0x0,0x4c(%rax)
crash> gdb list *pick_task_fair+71
0xffffffff810d5b57 is in pick_task_fair (kernel/sched/fair.c:5498).
5493 SCHED_WARN_ON(cfs_rq->next->sched_delayed);
5494 return cfs_rq->next;
5495 }
5496
5497 struct sched_entity *se = pick_eevdf(cfs_rq);
5498 if (se->sched_delayed) {
5499 dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
5500 SCHED_WARN_ON(se->sched_delayed);
5501 SCHED_WARN_ON(se->on_rq);
5502 if (sched_feat(DELAY_ZERO) && se->vlag > 0)
crash> struct -ox sched_entity

struct sched_entity {
    [0x0] struct load_weight load;

[0x10] struct rb_node run_node;
[0x28] u64 deadline;
[0x30] u64 min_vruntime;
[0x38] struct list_head group_node;
[0x48] unsigned int on_rq;
[0x4c] unsigned char sched_delayed;
[0x4d] unsigned char custom_slice;
[0x50] u64 exec_start;
[0x58] u64 sum_exec_runtime;
[0x60] u64 prev_sum_exec_runtime;
[0x68] u64 vruntime;
[0x70] s64 vlag;
[0x78] u64 slice;
[0x80] u64 nr_migrations;
[0xc0] struct sched_avg avg;
}
SIZE: 0x100
crash>

#17 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

(+ Chen Yu, Oliver Sang)

Hello Mike,

Quoted 77 lines · Mike Galbraith
Quoted 11 lines · K Prateek Nayak

I ran into a few issues when testing the series on top of tip:sched/core at commit 4475cd8bfd9b ("sched/balancing: Simplify the sg_status bitmask and use separate ->overloaded and ->overutilized flags"). All of these splats surfaced when running unixbench with Delayed Dequeue (echoing NO_DELAY_DEQUEUE to /sys/kernel/debug/sched/features seems to make the system stable when running Unixbench spawn)

Unixbench (https://github.com/kdlucas/byte-unixbench.git) command:

        ./Run spawn -c 512

That plus a hackbench loop works a treat.

Quoted 4 lines

Splats appear soon into the run. Following are the splats and their corresponding code blocks from my 3rd Generation EPYC system (2 x 64C/128T):

Seems a big box is not required. With a low fat sched config (no group sched), starting ./Run spawn -c 16 (cpus*2) along with a hackbench loop reliably blows my old i7-4790 box out of the water nearly instantly.

    DUMPFILE: vmcore
        CPUS: 8
        DATE: Mon Apr 15 07:20:29 CEST 2024
      UPTIME: 00:07:23
LOAD AVERAGE: 1632.20, 684.99, 291.84
       TASKS: 1401
    NODENAME: homer
     RELEASE: 6.9.0.g0bbac3f-master
     VERSION: #7 SMP Mon Apr 15 06:40:05 CEST 2024
     MACHINE: x86_64  (3591 Mhz)
      MEMORY: 16 GB
       PANIC: "Oops: 0000 [#1] SMP NOPTI" (check log for details)
         PID: 22664
     COMMAND: "hackbench"
        TASK: ffff888100acbf00  [THREAD_INFO: ffff888100acbf00]
         CPU: 5
       STATE: TASK_WAKING (PANIC)

crash> bt -sx
PID: 22664 TASK: ffff888100acbf00 CPU: 5 COMMAND: "hackbench"
#0 [ffff88817cc17920] machine_kexec+0x156 at ffffffff810642d6 #1 [ffff88817cc17970] __crash_kexec+0xd7 at ffffffff81153147 #2 [ffff88817cc17a28] crash_kexec+0x23 at ffffffff811535f3 #3 [ffff88817cc17a38] oops_end+0xbe at ffffffff810329be #4 [ffff88817cc17a58] page_fault_oops+0x81 at ffffffff81071951 #5 [ffff88817cc17ab8] exc_page_fault+0x62 at ffffffff8194f6f2 #6 [ffff88817cc17ae0] asm_exc_page_fault+0x22 at ffffffff81a00ba2

    [exception RIP: pick_task_fair+71]
    RIP: ffffffff810d5b57  RSP: ffff88817cc17b90  RFLAGS: 00010046
    RAX: 0000000000000000  RBX: ffff88840ed70ec0  RCX: 00000001d7ec138c
    RDX: ffffffffe7a7f400  RSI: 0000000000000000  RDI: 0000000000000000
    RBP: ffff88840ed70ec0   R8: 0000000000000c00   R9: 000000675402f79e
    R10: ffff88817cc17b30  R11: 00000000000000bb  R12: ffff88840ed70f40
    R13: ffffffff81f64f16  R14: ffff888100acc560  R15: ffff888100acbf00
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

#7 [ffff88817cc17bb0] pick_next_task_fair+0x42 at ffffffff810d92c2 #8 [ffff88817cc17be0] __schedule+0x10d at ffffffff8195936d
#9 [ffff88817cc17c50] schedule+0x1c at ffffffff81959ddc
#10 [ffff88817cc17c60] schedule_timeout+0x18c at ffffffff8195fc4c #11 [ffff88817cc17cc8] unix_stream_read_generic+0x2b7 at ffffffff81869917 #12 [ffff88817cc17da8] unix_stream_recvmsg+0x68 at ffffffff8186a2d8 #13 [ffff88817cc17de0] sock_read_iter+0x159 at ffffffff8170bd69 #14 [ffff88817cc17e70] vfs_read+0x2ce at ffffffff812f195e
#15 [ffff88817cc17ef8] ksys_read+0x40 at ffffffff812f21d0
#16 [ffff88817cc17f30] do_syscall_64+0x57 at ffffffff8194b947
#17 [ffff88817cc17f50] entry_SYSCALL_64_after_hwframe+0x76 at ffffffff81a0012b

    RIP: 00007f625660871e  RSP: 00007ffc75d48188  RFLAGS: 00000246
    RAX: ffffffffffffffda  RBX: 00007ffc75d48200  RCX: 00007f625660871e
    RDX: 0000000000000064  RSI: 00007ffc75d48190  RDI: 0000000000000007
    RBP: 00007ffc75d48260   R8: 00007ffc75d48140   R9: 00007f6256612010
    R10: 00007f62565f5070  R11: 0000000000000246  R12: 0000000000000064
    R13: 0000000000000000  R14: 0000000000000064  R15: 0000000000000000
    ORIG_RAX: 0000000000000000  CS: 0033  SS: 002b
Quoted 10 lines

crash> dis pick_task_fair+71
0xffffffff810d5b57 <pick_task_fair+71>: cmpb $0x0,0x4c(%rax)
crash> gdb list *pick_task_fair+71
0xffffffff810d5b57 is in pick_task_fair (kernel/sched/fair.c:5498).
5493 SCHED_WARN_ON(cfs_rq->next->sched_delayed);
5494 return cfs_rq->next;
5495 }
5496
5497 struct sched_entity *se = pick_eevdf(cfs_rq);
5498 if (se->sched_delayed) {

Wondering if you are running into the issue where pick_eevdf() returns NULL despite there being runnable CFS task on the runqueue. Can you try running with the following patch from Chenyu -
https://lore.kernel.org/lkml/[email protected]/

Quoted 23 lines

5499 dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
5500 SCHED_WARN_ON(se->sched_delayed);
5501 SCHED_WARN_ON(se->on_rq);
5502 if (sched_feat(DELAY_ZERO) && se->vlag > 0)
crash> struct -ox sched_entity

struct sched_entity {
    [0x0] struct load_weight load;
   [0x10] struct rb_node run_node;
   [0x28] u64 deadline;
   [0x30] u64 min_vruntime;
   [0x38] struct list_head group_node;
   [0x48] unsigned int on_rq;
   [0x4c] unsigned char sched_delayed;
   [0x4d] unsigned char custom_slice;
   [0x50] u64 exec_start;
   [0x58] u64 sum_exec_runtime;
   [0x60] u64 prev_sum_exec_runtime;
   [0x68] u64 vruntime;
   [0x70] s64 vlag;
   [0x78] u64 slice;
   [0x80] u64 nr_migrations;
   [0xc0] struct sched_avg avg;
}
Quoted 3 lines

SIZE: 0x100
crash>

--
Thanks and Regards,
Prateek

#18 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 5 lines · K Prateek Nayak

Wondering if you are running into the issue where pick_eevdf() returns NULL despite there being runnable CFS task on the runqueue. Can you try running with the following patch from Chenyu -
https://lore.kernel.org/lkml/[email protected]/

I hadn't tried it in master branch, but had in my 6.1-eevdf tree which contains the previous delay dequeue version, because I found that it too would explode, which surprised me having run it quite a bit woe free. Patch helped 6.1-eevdf not at all. I just tried it in master to be sure, and while the death throes were not identical, this one began with SCHED_WARN_ON() in put_prev_entity(), box still went belly up.

BTW/FWIW, turning on AUTOGROUP seemingly rendered box immune (unless booted with noautogroup 'course), and that immunity even held when I launched all players in the same cgroup (hmm).

-Mike

#19 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Greetings,

I tossed a couple rocks at it today, and seem to have hit the little bugger. The root cause seems to be doing the delay dequeue business on exiting tasks. Hunk #1 of hacklet below seems to quell the explosions.

crash> bt -sx
PID: 21722 TASK: ffff88815710ee40 CPU: 6 COMMAND: "hackbench"
#0 [ffff88822773bb90] machine_kexec+0x156 at ffffffff810642d6 #1 [ffff88822773bbe0] __crash_kexec+0xd7 at ffffffff81152a07 #2 [ffff88822773bc98] crash_kexec+0x23 at ffffffff81152eb3 #3 [ffff88822773bca8] oops_end+0xbe at ffffffff810329be #4 [ffff88822773bcc8] page_fault_oops+0x81 at ffffffff81071951 #5 [ffff88822773bd28] exc_page_fault+0x62 at ffffffff8194e9e2 #6 [ffff88822773bd50] asm_exc_page_fault+0x22 at ffffffff81a00ba2

    [exception RIP: pick_next_task_fair+178]
    RIP: ffffffff810d8d12  RSP: ffff88822773be00  RFLAGS: 00010006
    RAX: ffff88813cb780b8  RBX: ffff88840edb0e80  RCX: 0000000000000000
    RDX: 0000000000000000  RSI: ffff88813cb78080  RDI: ffff88840ec30f00
    RBP: ffff88813cb78000   R8: ffff88815710eec0   R9: 0000000000000001
    R10: ffff88822773bdc8  R11: 0000000000000013  R12: 0000000000030e80
    R13: ffff88815710ee40  R14: ffff88813cb78080  R15: ffff88815710ee40
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

#7 [ffff88822773be28] __schedule+0x10d at ffffffff8195865d
#8 [ffff88822773be98] do_task_dead+0x3e at ffffffff810cc00e
#9 [ffff88822773beb0] do_exit+0x770 at ffffffff8108f0e0
#10 [ffff88822773bf00] do_group_exit+0x2c at ffffffff8108f64c
#11 [ffff88822773bf28] __x64_sys_exit_group+0x14 at ffffffff8108f6f4 #12 [ffff88822773bf30] do_syscall_64+0x57 at ffffffff8194ac37
#13 [ffff88822773bf50] entry_SYSCALL_64_after_hwframe+0x76 at ffffffff81a0012b

    RIP: 00007f4f2aa76136  RSP: 00007ffcbba84748  RFLAGS: 00000246
    RAX: ffffffffffffffda  RBX: 0000000000000000  RCX: 00007f4f2aa76136
    RDX: 0000000000000000  RSI: 000000000000003c  RDI: 0000000000000000
    RBP: 00007f4f2ab86970   R8: 00000000000000e7   R9: ffffffffffffff80
    R10: 0000000000000004  R11: 0000000000000246  R12: 00007f4f2ab86970
    R13: 0000000000000001  R14: 00007f4f2ab8a328  R15: 0000000000000000
    ORIG_RAX: 00000000000000e7  CS: 0033  SS: 002b
crash> task_struct ffff88815710ee40 | grep sched_delayed
    sched_delayed = 1 '\001',

crash>

---
 kernel/sched/fair.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/fair.c +3 -2
@@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
 		update_curr(cfs_rq);
 
 		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
 		    !entity_eligible(cfs_rq, se)) {
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;
@@ -5495,14 +5496,14 @@ pick_next_entity(struct rq *rq, struct c
 	}
 
 	struct sched_entity *se = pick_eevdf(cfs_rq);
-	if (se->sched_delayed) {
+	while (se && se->sched_delayed) {
 		dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
 		SCHED_WARN_ON(se->sched_delayed);
 		SCHED_WARN_ON(se->on_rq);
 		if (sched_feat(DELAY_ZERO) && se->vlag > 0)
 			se->vlag = 0;
 
-		return NULL;
+		se = pick_eevdf(cfs_rq);
 	}
 	return se;
 }
#20 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Hello Mike,

Quoted 39 lines · Mike Galbraith

Greetings,

I tossed a couple rocks at it today, and seem to have hit the little bugger. The root cause seems to be doing the delay dequeue business on exiting tasks. Hunk #1 of hacklet below seems to quell the explosions.

crash> bt -sx
PID: 21722 TASK: ffff88815710ee40 CPU: 6 COMMAND: "hackbench"
#0 [ffff88822773bb90] machine_kexec+0x156 at ffffffff810642d6 #1 [ffff88822773bbe0] __crash_kexec+0xd7 at ffffffff81152a07 #2 [ffff88822773bc98] crash_kexec+0x23 at ffffffff81152eb3 #3 [ffff88822773bca8] oops_end+0xbe at ffffffff810329be #4 [ffff88822773bcc8] page_fault_oops+0x81 at ffffffff81071951 #5 [ffff88822773bd28] exc_page_fault+0x62 at ffffffff8194e9e2 #6 [ffff88822773bd50] asm_exc_page_fault+0x22 at ffffffff81a00ba2

    [exception RIP: pick_next_task_fair+178]
    RIP: ffffffff810d8d12  RSP: ffff88822773be00  RFLAGS: 00010006
    RAX: ffff88813cb780b8  RBX: ffff88840edb0e80  RCX: 0000000000000000
    RDX: 0000000000000000  RSI: ffff88813cb78080  RDI: ffff88840ec30f00
    RBP: ffff88813cb78000   R8: ffff88815710eec0   R9: 0000000000000001
    R10: ffff88822773bdc8  R11: 0000000000000013  R12: 0000000000030e80
    R13: ffff88815710ee40  R14: ffff88813cb78080  R15: ffff88815710ee40
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

#7 [ffff88822773be28] __schedule+0x10d at ffffffff8195865d
#8 [ffff88822773be98] do_task_dead+0x3e at ffffffff810cc00e
#9 [ffff88822773beb0] do_exit+0x770 at ffffffff8108f0e0
#10 [ffff88822773bf00] do_group_exit+0x2c at ffffffff8108f64c
#11 [ffff88822773bf28] __x64_sys_exit_group+0x14 at ffffffff8108f6f4 #12 [ffff88822773bf30] do_syscall_64+0x57 at ffffffff8194ac37
#13 [ffff88822773bf50] entry_SYSCALL_64_after_hwframe+0x76 at ffffffff81a0012b

    RIP: 00007f4f2aa76136  RSP: 00007ffcbba84748  RFLAGS: 00000246
    RAX: ffffffffffffffda  RBX: 0000000000000000  RCX: 00007f4f2aa76136
    RDX: 0000000000000000  RSI: 000000000000003c  RDI: 0000000000000000
    RBP: 00007f4f2ab86970   R8: 00000000000000e7   R9: ffffffffffffff80
    R10: 0000000000000004  R11: 0000000000000246  R12: 00007f4f2ab86970
    R13: 0000000000000001  R14: 00007f4f2ab8a328  R15: 0000000000000000
    ORIG_RAX: 00000000000000e7  CS: 0033  SS: 002b
crash> task_struct ffff88815710ee40 | grep sched_delayed
    sched_delayed = 1 '\001',
Quoted 2 lines

crash>

My machine too survived the spawn test with the below patch. Thank you for looking into this. I'll resume testing with the below patch included.

--
Thanks and Regards,
Prateek

Quoted 32 lines
---
 kernel/sched/fair.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/fair.c +3 -2
@@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
 		update_curr(cfs_rq);
 
 		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
 		    !entity_eligible(cfs_rq, se)) {
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;
@@ -5495,14 +5496,14 @@ pick_next_entity(struct rq *rq, struct c
 	}
 
 	struct sched_entity *se = pick_eevdf(cfs_rq);
-	if (se->sched_delayed) {
+	while (se && se->sched_delayed) {
 		dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
 		SCHED_WARN_ON(se->sched_delayed);
 		SCHED_WARN_ON(se->on_rq);
 		if (sched_feat(DELAY_ZERO) && se->vlag > 0)
 			se->vlag = 0;
 
-		return NULL;
+		se = pick_eevdf(cfs_rq);
 	}
 	return se;
 }
author maintainer
#21 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Mike Galbraith
Quoted 6 lines · Mike Galbraith

Greetings,

I tossed a couple rocks at it today, and seem to have hit the little bugger. The root cause seems to be doing the delay dequeue business on exiting tasks. Hunk #1 of hacklet below seems to quell the explosions.

Neat, Thanks! I was just about ready to go look at this again.

Quoted 33 lines
---
 kernel/sched/fair.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/fair.c +3 -2
@@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
 		update_curr(cfs_rq);
 
 		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
 		    !entity_eligible(cfs_rq, se)) {
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;
@@ -5495,14 +5496,14 @@ pick_next_entity(struct rq *rq, struct c
 	}
 
 	struct sched_entity *se = pick_eevdf(cfs_rq);
-	if (se->sched_delayed) {
+	while (se && se->sched_delayed) {
 		dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
 		SCHED_WARN_ON(se->sched_delayed);
 		SCHED_WARN_ON(se->on_rq);
 		if (sched_feat(DELAY_ZERO) && se->vlag > 0)
 			se->vlag = 0;
 
-		return NULL;
+		se = pick_eevdf(cfs_rq);
 	}
 	return se;
 }
author maintainer
#22 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Mike Galbraith
Quoted 10 lines · Mike Galbraith

The root cause seems to be doing the delay dequeue business on exiting tasks.

---
 kernel/sched/fair.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

kernel/sched/fair.c dequeue_entity(struct cfs_rq *cfs_rq, st · line 5380

Quoted 7 lines
kernel/sched/fair.c +1 -0
@@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
 		update_curr(cfs_rq);
 
 		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
 		    !entity_eligible(cfs_rq, se)) {
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;

So I think this can be easier done in dequeue_task_fair(), where we still know this is a task.

Perhaps something like (I'll test later):

	if (p->flags & PF_EXITING)
		flags &= ~DEQUEUE_SLEEP;

But now I need to go think about the case of removing a cgroup... *urgh*.

author maintainer
#23 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 22 lines · Peter Zijlstra
Quoted 10 lines · Mike Galbraith

The root cause seems to be doing the delay dequeue business on exiting tasks.

---
 kernel/sched/fair.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

kernel/sched/fair.c dequeue_entity(struct cfs_rq *cfs_rq, st · line 5380

Quoted 7 lines
kernel/sched/fair.c +1 -0
@@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
 		update_curr(cfs_rq);
 
 		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
 		    !entity_eligible(cfs_rq, se)) {
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;

So I think this can be easier done in dequeue_task_fair(), where we still know this is a task.

Quoted 7 lines

Perhaps something like (I'll test later):

	if (p->flags & PF_EXITING)
		flags &= ~DEQUEUE_SLEEP;

But now I need to go think about the case of removing a cgroup... *urgh*.

I ended up with the below instead; lemme go run this unixbench spawn on it.

---

kernel/sched/fair.c +32 -13
index 95666034e76c..b5918fa9a0f0 100644
@@ -8429,7 +8431,20 @@ static void migrate_task_rq_fair(struct task_struct *p, int new_cpu)
 
 static void task_dead_fair(struct task_struct *p)
 {
-	remove_entity_load_avg(&p->se);
+	struct sched_entity *se = &p->se;
+
+	if (p->se.sched_delayed) {
+		struct rq_flags rf;
+		struct rq *rq;
+
+		rq = task_rq_lock(p, &rf);
+		update_rq_clock(rq);
+		if (se->sched_delayed)
+			dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
+		task_rq_unlock(rq, p, &rf);
+	}
+
+	remove_entity_load_avg(se);
 }
 
 /*
@@ -13089,28 +13104,34 @@ void online_fair_sched_group(struct task_group *tg)
 
 void unregister_fair_sched_group(struct task_group *tg)
 {
-	unsigned long flags;
-	struct rq *rq;
 	int cpu;
 
 	destroy_cfs_bandwidth(tg_cfs_bandwidth(tg));
 
 	for_each_possible_cpu(cpu) {
-		if (tg->se[cpu])
-			remove_entity_load_avg(tg->se[cpu]);
+		struct cfs_rq *cfs_rq = tg->cfs_rq[cpu];
+		struct sched_entity *se = tg->se[cpu];
+		struct rq *rq = cpu_rq(cpu);
+
+		if (se) {
+			if (se->sched_delayed) {
+				guard(rq_lock_irqsave)(rq);
+				update_rq_clock(rq);
+				if (se->sched_delayed)
+					dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
+				list_del_leaf_cfs_rq(cfs_rq);
+			}
+			remove_entity_load_avg(se);
+		}
 
 		/*
 		 * Only empty task groups can be destroyed; so we can speculatively
 		 * check on_list without danger of it being re-added.
 		 */
-		if (!tg->cfs_rq[cpu]->on_list)
-			continue;
-
-		rq = cpu_rq(cpu);
-
-		raw_spin_rq_lock_irqsave(rq, flags);
-		list_del_leaf_cfs_rq(tg->cfs_rq[cpu]);
-		raw_spin_rq_unlock_irqrestore(rq, flags);
+		if (cfs_rq->on_list) {
+			guard(rq_lock_irqsave)(rq);
+			list_del_leaf_cfs_rq(cfs_rq);
+		}
 	}
 }
 
author maintainer
#24 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 25 lines · Peter Zijlstra
Quoted 24 lines · Peter Zijlstra
Quoted 10 lines · Mike Galbraith

The root cause seems to be doing the delay dequeue business on exiting tasks.

---
 kernel/sched/fair.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

kernel/sched/fair.c dequeue_entity(struct cfs_rq *cfs_rq, st · line 5380

Quoted 7 lines
kernel/sched/fair.c +1 -0
@@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
 		update_curr(cfs_rq);
 
 		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
 		    !entity_eligible(cfs_rq, se)) {
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;

So I think this can be easier done in dequeue_task_fair(), where we still know this is a task.

Perhaps something like (I'll test later):

Quoted 7 lines
Quoted 5 lines
	if (p->flags & PF_EXITING)
		flags &= ~DEQUEUE_SLEEP;

But now I need to go think about the case of removing a cgroup... *urgh*.

I ended up with the below instead; lemme go run this unixbench spawn on it.

Seems to survive that.

I pushed out the patches with updates to queue/sched/eevdf

#25 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

I ended up with the below instead; lemme go run this unixbench spawn on it.

Seems to survive that.

I pushed out the patches with updates to queue/sched/eevdf

Yup, solid... but fwiw, tbench liked the previous version better.

trusty (with a 'c') ole i7-4790 box, tbench 8

for i in 1 2 3; do tbench.sh 8 10 2>&1|grep Throughput; done

6.9.0.gc942a0c-master +eevdf.current
NO_DELAY_DEQUEUE
Throughput 3285.04 MB/sec 8 clients 8 procs max_latency=3.481 ms Throughput 3289.66 MB/sec 8 clients 8 procs max_latency=8.124 ms Throughput 3293.83 MB/sec 8 clients 8 procs max_latency=2.210 ms DELAY_DEQUEUE
Throughput 3246.3 MB/sec 8 clients 8 procs max_latency=2.181 ms Throughput 3236.96 MB/sec 8 clients 8 procs max_latency=6.988 ms Throughput 3248.6 MB/sec 8 clients 8 procs max_latency=2.130 ms

6.9.0.gc942a0c-master +eevdf.prev
NO_DELAY_DEQUEUE
Throughput 3457.92 MB/sec 8 clients 8 procs max_latency=3.885 ms Throughput 3470.95 MB/sec 8 clients 8 procs max_latency=4.475 ms Throughput 3467.87 MB/sec 8 clients 8 procs max_latency=2.182 ms DELAY_DEQUEUE
Throughput 3712.96 MB/sec 8 clients 8 procs max_latency=4.231 ms Throughput 3667.87 MB/sec 8 clients 8 procs max_latency=5.020 ms Throughput 3679.65 MB/sec 8 clients 8 procs max_latency=2.847 ms

Trees are identical modulo extracted eevdf additions. The previous win that put eevdf on par with cfs went missing.. and then some.

For reference, cfs vs eevdf log extract for previously mentioned gain.

6.1.87-cfs

Throughput 3660.98 MB/sec  8 clients  8 procs  max_latency=2.204 ms
Throughput 3678.67 MB/sec  8 clients  8 procs  max_latency=10.127 ms
Throughput 3631.89 MB/sec  8 clients  8 procs  max_latency=13.019 ms
              1.000

6.1.87-eevdf - naked eevdf +fixes

Throughput 3441.86 MB/sec  8 clients  8 procs  max_latency=3.943 ms
Throughput 3439.68 MB/sec  8 clients  8 procs  max_latency=4.285 ms
Throughput 3432.28 MB/sec  8 clients  8 procs  max_latency=3.557 ms

vs cfs .940

6.1.87-eevdf +delay_dequeue.prev patch set
DELAY_DEQUEUE

Throughput 3696.94 MB/sec  8 clients  8 procs  max_latency=2.179 ms
Throughput 3694.64 MB/sec  8 clients  8 procs  max_latency=6.322 ms
Throughput 3654.49 MB/sec  8 clients  8 procs  max_latency=4.101 ms

vs cfs 1.006

box waxes nostalgic (son, when I was yo age [flex]) 4.19.312

Throughput 4099.07 MB/sec  8 clients  8 procs  max_latency=2.169 ms
Throughput 4107.49 MB/sec  8 clients  8 procs  max_latency=12.404 ms
Throughput 4118.41 MB/sec  8 clients  8 procs  max_latency=14.150 ms

-Mike

#26 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 1 line · Mike Galbraith

fwiw, tbench liked the previous version better.

The culprit lies somewhere in the new PREEMPT_SHORT patch.

-Mike

#27 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 4 lines · Mike Galbraith
Quoted 1 line · Mike Galbraith

fwiw, tbench liked the previous version better.

The culprit lies somewhere in the new PREEMPT_SHORT patch.

<squint>

Ah, moving the curr eligibility check into pick_curr() left an ineligible curr to be seen/used further down in pick_eevdf().

found:

        if (!best || (curr && entity_before(curr, best)))
                best = curr;

Nit: PREEMPT_SHORT depending on RUN_TO_PARITY looks odd.

-Mike

author maintainer
#28 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 6 lines · Mike Galbraith
Quoted 4 lines · Mike Galbraith
Quoted 1 line · Mike Galbraith

fwiw, tbench liked the previous version better.

The culprit lies somewhere in the new PREEMPT_SHORT patch.

Quoted 8 lines

<squint>

Ah, moving the curr eligibility check into pick_curr() left an ineligible curr to be seen/used further down in pick_eevdf().

found:

        if (!best || (curr && entity_before(curr, best)))
                best = curr;

Hmm yes, over aggressive cleanup that. Let me try again.

Quoted 1 line

Nit: PREEMPT_SHORT depending on RUN_TO_PARITY looks odd.

The thinking was that without RUN_TO_PARITY we'll always do a full pick and it will always pick a (new) shorter deadline task over current.

The PREEMPRT_SHORT thing really is an exception to avoid RUN_TO_PARITY from ruining latency game for short tasks.

#29 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra

Hi Peter,

Quoted 19 lines · Peter Zijlstra

Extend / fix 86bfbb7ce4f6 ("sched/fair: Add lag based placement") by noting that lag is fundamentally a temporal measure. It should not be carried around indefinitely.

OTOH it should also not be instantly discarded, doing so will allow a task to game the system by purposefully (micro) sleeping at the end of its time quantum.

Since lag is intimately tied to the virtual time base, a wall-time based decay is also insufficient, notably competition is required for any of this to make sense.

Instead, delay the dequeue and keep the 'tasks' on the runqueue, competing until they are eligible.

Strictly speaking, we only care about keeping them until the 0-lag point, but that is a difficult proposition, instead carry them around until they get picked again, and dequeue them at that point.

Quoted 6 lines

Since we should have dequeued them at the 0-lag point, truncate lag (eg. don't let them earn positive lag).

XXX test the cfs-throttle stuff

Playing around with a Pixel 6 running a 6.6-based kernel with this series backported on top, I spotted a noticeable performance improvement on the speedometer benchmark:

- m6.6-stock-* is the 6.6 mainline Android kernel unmodified.

- m6.6-eevdf-complete-* is the 6.6 mainline Android kernel with this series applied on top (along with a few required backported patches).

+-------------------+-----------------------+-----------+
|      metric       |          tag          | perc_diff |
+-------------------+-----------------------+-----------+
| Speedometer Score |     m6.6-stock-1      |   0.0%    |
| Speedometer Score |     m6.6-stock-2      |   1.23%   |
| Speedometer Score |     m6.6-stock-3      |  -0.22%   |
| Speedometer Score | m6.6-eevdf-complete-1 |   4.54%   |
| Speedometer Score | m6.6-eevdf-complete-2 |   4.95%   |
| Speedometer Score | m6.6-eevdf-complete-3 |   6.07%   |
+-------------------+-----------------------+-----------+

Also some interesting improvements in terms of frame timing for the uibenchjanktests benchmark. In particular the metrics of missed deadlines and jank (late) frames, which seems to indicate better latencies.

+-----------------------+-----------------------+-----------+
|        metric         |          tag          | perc_diff |
+-----------------------+-----------------------+-----------+
| gfx-avg-frame-time-50 |     m6.6-stock-1      |    0.0    |
| gfx-avg-frame-time-90 |     m6.6-stock-1      |    0.0    |
| gfx-avg-frame-time-95 |     m6.6-stock-1      |    0.0    |
| gfx-avg-frame-time-99 |     m6.6-stock-1      |    0.0    |
| gfx-avg-frame-time-50 |     m6.6-stock-2      |   3.46    |
| gfx-avg-frame-time-90 |     m6.6-stock-2      |   1.19    |
| gfx-avg-frame-time-95 |     m6.6-stock-2      |   0.24    |
| gfx-avg-frame-time-99 |     m6.6-stock-2      |   0.48    |
| gfx-avg-frame-time-50 | m6.6-eevdf-complete-1 |  -30.45   |
| gfx-avg-frame-time-90 | m6.6-eevdf-complete-1 |  -48.44   |
| gfx-avg-frame-time-95 | m6.6-eevdf-complete-1 |  -51.32   |
| gfx-avg-frame-time-99 | m6.6-eevdf-complete-1 |  -52.48   |
| gfx-avg-frame-time-50 | m6.6-eevdf-complete-2 |  -30.32   |
| gfx-avg-frame-time-90 | m6.6-eevdf-complete-2 |  -48.16   |
| gfx-avg-frame-time-95 | m6.6-eevdf-complete-2 |  -51.08   |
| gfx-avg-frame-time-99 | m6.6-eevdf-complete-2 |   -51.7   |
+-----------------------+-----------------------+-----------+
+-----------------------------------+-----------------------+-----------+
|              metric               |          tag          | perc_diff |
+-----------------------------------+-----------------------+-----------+
| gfx-avg-num-frame-deadline-missed |     m6.6-stock-1      |    0.0    |
| gfx-max-num-frame-deadline-missed |     m6.6-stock-1      |    0.0    |
| gfx-avg-num-frame-deadline-missed |     m6.6-stock-2      |   -3.21   |
| gfx-max-num-frame-deadline-missed |     m6.6-stock-2      |   -3.21   |
| gfx-avg-num-frame-deadline-missed | m6.6-eevdf-complete-1 |  -85.29   |
| gfx-max-num-frame-deadline-missed | m6.6-eevdf-complete-1 |  -85.29   |
| gfx-avg-num-frame-deadline-missed | m6.6-eevdf-complete-2 |   -84.8   |
| gfx-max-num-frame-deadline-missed | m6.6-eevdf-complete-2 |   -84.8   |
+-----------------------------------+-----------------------+-----------+
+----------------------------+-----------------------+-----------+
|           metric           |          tag          | perc_diff |
+----------------------------+-----------------------+-----------+
| gfx-avg-high-input-latency |     m6.6-stock-1      |    0.0    |
| gfx-max-high-input-latency |     m6.6-stock-1      |    0.0    |
| gfx-avg-high-input-latency |     m6.6-stock-2      |   0.93    |
| gfx-max-high-input-latency |     m6.6-stock-2      |   0.93    |
| gfx-avg-high-input-latency | m6.6-eevdf-complete-1 |  -18.35   |
| gfx-max-high-input-latency | m6.6-eevdf-complete-1 |  -18.35   |
| gfx-avg-high-input-latency | m6.6-eevdf-complete-2 |  -18.05   |
| gfx-max-high-input-latency | m6.6-eevdf-complete-2 |  -18.05   |
+----------------------------+-----------------------+-----------+
+--------------+-----------------------+-----------+
|    metric    |          tag          | perc_diff |
+--------------+-----------------------+-----------+
| gfx-avg-jank |     m6.6-stock-1      |    0.0    |
| gfx-max-jank |     m6.6-stock-1      |    0.0    |
| gfx-avg-jank |     m6.6-stock-2      |   1.56    |
| gfx-max-jank |     m6.6-stock-2      |   1.56    |
| gfx-avg-jank | m6.6-eevdf-complete-1 |  -82.81   |
| gfx-max-jank | m6.6-eevdf-complete-1 |  -82.81   |
| gfx-avg-jank | m6.6-eevdf-complete-2 |  -78.12   |
| gfx-max-jank | m6.6-eevdf-complete-2 |  -78.12   |
+--------------+-----------------------+-----------+

Bisecting through the patches in this series, I ended up with patch 08/10 as the one that improved things overall for these benchmarks.

I'd like to investigate this further to understand the reason behind some of these dramatic improvements.

Quoted 417 lines
---
 include/linux/sched.h   |    1 
 kernel/sched/core.c     |   22 +++++--
 kernel/sched/fair.c     |  148 +++++++++++++++++++++++++++++++++++++++++++-----
 kernel/sched/features.h |   12 +++
 kernel/sched/sched.h    |    2 
 5 files changed, 167 insertions(+), 18 deletions(-)
include/linux/sched.h +1 -0
@@ -542,6 +542,7 @@ struct sched_entity {
 
 	struct list_head		group_node;
 	unsigned int			on_rq;
+	unsigned int			sched_delayed;
 
 	u64				exec_start;
 	u64				sum_exec_runtime;
kernel/sched/core.c +18 -4
@@ -2154,10 +2154,14 @@ void activate_task(struct rq *rq, struct
 
 void deactivate_task(struct rq *rq, struct task_struct *p, int flags)
 {
-	WRITE_ONCE(p->on_rq, (flags & DEQUEUE_SLEEP) ? 0 : TASK_ON_RQ_MIGRATING);
-	ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+	bool sleep = flags & DEQUEUE_SLEEP;
 
-	dequeue_task(rq, p, flags);
+	if (dequeue_task(rq, p, flags)) {
+		WRITE_ONCE(p->on_rq, sleep ? 0 : TASK_ON_RQ_MIGRATING);
+		ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+	} else {
+		SCHED_WARN_ON(!sleep); /* only sleep can fail */
+	}
 }
 
 static inline int __normal_prio(int policy, int rt_prio, int nice)
@@ -3858,12 +3862,17 @@ static int ttwu_runnable(struct task_str
 
 	rq = __task_rq_lock(p, &rf);
 	if (task_on_rq_queued(p)) {
+		update_rq_clock(rq);
+		if (p->se.sched_delayed) {
+			/* mustn't run a delayed task */
+			SCHED_WARN_ON(task_on_cpu(rq, p));
+			enqueue_task(rq, p, ENQUEUE_DELAYED);
+		}
 		if (!task_on_cpu(rq, p)) {
 			/*
 			 * When on_rq && !on_cpu the task is preempted, see if
 			 * it should preempt the task that is current now.
 			 */
-			update_rq_clock(rq);
 			wakeup_preempt(rq, p, wake_flags);
 		}
 		ttwu_do_wakeup(p);
@@ -4243,11 +4252,16 @@ int try_to_wake_up(struct task_struct *p
 		 * case the whole 'p->on_rq && ttwu_runnable()' case below
 		 * without taking any locks.
 		 *
+		 * Specifically, given current runs ttwu() we must be before
+		 * schedule()'s deactivate_task(), as such this must not
+		 * observe sched_delayed.
+		 *
 		 * In particular:
 		 *  - we rely on Program-Order guarantees for all the ordering,
 		 *  - we're serialized against set_special_state() by virtue of
 		 *    it disabling IRQs (this allows not taking ->pi_lock).
 		 */
+		SCHED_WARN_ON(p->se.sched_delayed);
 		if (!ttwu_state_match(p, state, &success))
 			goto out;
 
kernel/sched/fair.c +134 -14
@@ -5270,6 +5270,10 @@ static inline int cfs_rq_throttled(struc
 static inline bool cfs_bandwidth_used(void);
 
 static void
+requeue_delayed_entity(struct sched_entity *se);
+
+/* XXX bool and pull in the requeue_delayed_entity thing */
+static void
 enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
 {
 	bool curr = cfs_rq->curr == se;
@@ -5356,18 +5360,33 @@ static void clear_buddies(struct cfs_rq
 
 static __always_inline void return_cfs_rq_runtime(struct cfs_rq *cfs_rq);
 
-static void
+static bool
 dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
 {
-	int action = UPDATE_TG;
-
-	if (entity_is_task(se) && task_on_rq_migrating(task_of(se)))
-		action |= DO_DETACH;
-
 	/*
 	 * Update run-time statistics of the 'current'.
 	 */
-	update_curr(cfs_rq);
+	if (flags & DEQUEUE_DELAYED) {
+		SCHED_WARN_ON(!se->sched_delayed);
+		se->sched_delayed = 0;
+	} else {
+		bool sleep = flags & DEQUEUE_SLEEP;
+
+		SCHED_WARN_ON(sleep && se->sched_delayed);
+		update_curr(cfs_rq);
+
+		if (sched_feat(DELAY_DEQUEUE) && sleep &&
+		    !entity_eligible(cfs_rq, se)) {
+			if (cfs_rq->next == se)
+				cfs_rq->next = NULL;
+			se->sched_delayed = 1;
+			return false;
+		}
+	}
+
+	int action = UPDATE_TG;
+	if (entity_is_task(se) && task_on_rq_migrating(task_of(se)))
+		action |= DO_DETACH;
 
 	/*
 	 * When dequeuing a sched_entity, we must:
@@ -5407,6 +5426,8 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
 
 	if (cfs_rq->nr_running == 0)
 		update_idle_cfs_rq_clock_pelt(cfs_rq);
+
+	return true;
 }
 
 static void
@@ -5432,6 +5453,7 @@ set_next_entity(struct cfs_rq *cfs_rq, s
 	}
 
 	update_stats_curr_start(cfs_rq, se);
+	SCHED_WARN_ON(cfs_rq->curr);
 	cfs_rq->curr = se;
 
 	/*
@@ -5452,6 +5474,8 @@ set_next_entity(struct cfs_rq *cfs_rq, s
 	se->prev_sum_exec_runtime = se->sum_exec_runtime;
 }
 
+static int dequeue_entities(struct rq *rq, struct sched_entity *se, int flags);
+
 /*
  * Pick the next process, keeping these things in mind, in this order:
  * 1) keep things fair between processes/task groups
@@ -5460,16 +5484,29 @@ set_next_entity(struct cfs_rq *cfs_rq, s
  * 4) do not run the "skip" process, if something else is available
  */
 static struct sched_entity *
-pick_next_entity(struct cfs_rq *cfs_rq)
+pick_next_entity(struct rq *rq, struct cfs_rq *cfs_rq)
 {
 	/*
 	 * Enabling NEXT_BUDDY will affect latency but not fairness.
 	 */
 	if (sched_feat(NEXT_BUDDY) &&
-	    cfs_rq->next && entity_eligible(cfs_rq, cfs_rq->next))
+	    cfs_rq->next && entity_eligible(cfs_rq, cfs_rq->next)) {
+		/* ->next will never be delayed */
+		SCHED_WARN_ON(cfs_rq->next->sched_delayed);
 		return cfs_rq->next;
+	}
 
-	return pick_eevdf(cfs_rq);
+	struct sched_entity *se = pick_eevdf(cfs_rq);
+	if (se->sched_delayed) {
+		dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
+		SCHED_WARN_ON(se->sched_delayed);
+		SCHED_WARN_ON(se->on_rq);
+		if (sched_feat(DELAY_ZERO) && se->vlag > 0)
+			se->vlag = 0;
+
+		return NULL;
+	}
+	return se;
 }
 
 static bool check_cfs_rq_runtime(struct cfs_rq *cfs_rq);
@@ -5493,6 +5530,7 @@ static void put_prev_entity(struct cfs_r
 		/* in !on_rq case, update occurred at dequeue */
 		update_load_avg(cfs_rq, prev, 0);
 	}
+	SCHED_WARN_ON(cfs_rq->curr != prev);
 	cfs_rq->curr = NULL;
 }
 
@@ -5793,6 +5831,10 @@ static bool throttle_cfs_rq(struct cfs_r
 		if (!se->on_rq)
 			goto done;
 
+		/*
+		 * XXX should be fine vs sched_delay; if won't run after this.
+		 * Either pick dequeues it, or unthrottle. Double check!!
+		 */
 		dequeue_entity(qcfs_rq, se, DEQUEUE_SLEEP);
 
 		if (cfs_rq_is_idle(group_cfs_rq(se)))
@@ -5882,8 +5924,11 @@ void unthrottle_cfs_rq(struct cfs_rq *cf
 	for_each_sched_entity(se) {
 		struct cfs_rq *qcfs_rq = cfs_rq_of(se);
 
-		if (se->on_rq)
+		if (se->on_rq) {
+			if (se->sched_delayed)
+				requeue_delayed_entity(se);
 			break;
+		}
 		enqueue_entity(qcfs_rq, se, ENQUEUE_WAKEUP);
 
 		if (cfs_rq_is_idle(group_cfs_rq(se)))
@@ -6729,6 +6774,40 @@ static int sched_idle_cpu(int cpu)
 }
 #endif
 
+static void
+requeue_delayed_entity(struct sched_entity *se)
+{
+	struct cfs_rq *cfs_rq = cfs_rq_of(se);
+
+	/*
+	 * se->sched_delayed should imply both: se->on_rq == 1 and
+	 * cfs_rq->curr != se. Because a delayed entity is one that is still on
+	 * the runqueue competing until elegibility.
+	 *
+	 * Except for groups, consider current going idle and newidle pulling a
+	 * task in the same group -- in that case 'cfs_rq->curr == se'.
+	 */
+	SCHED_WARN_ON(!se->sched_delayed);
+	SCHED_WARN_ON(!se->on_rq);
+	SCHED_WARN_ON(entity_is_task(se) && cfs_rq->curr == se);
+
+	if (sched_feat(DELAY_ZERO)) {
+		update_entity_lag(cfs_rq, se);
+		if (se->vlag > 0) {
+			cfs_rq->nr_running--;
+			if (se != cfs_rq->curr)
+				__dequeue_entity(cfs_rq, se);
+			se->vlag = 0;
+			place_entity(cfs_rq, se, 0);
+			if (se != cfs_rq->curr)
+				__enqueue_entity(cfs_rq, se);
+			cfs_rq->nr_running++;
+		}
+	}
+
+	se->sched_delayed = 0;
+}
+
 /*
  * The enqueue_task method is called before nr_running is
  * increased. Here we update the fair scheduling stats and
@@ -6742,6 +6821,11 @@ enqueue_task_fair(struct rq *rq, struct
 	int idle_h_nr_running = task_has_idle_policy(p);
 	int task_new = !(flags & ENQUEUE_WAKEUP);
 
+	if (flags & ENQUEUE_DELAYED) {
+		requeue_delayed_entity(se);
+		return;
+	}
+
 	/*
 	 * The code below (indirectly) updates schedutil which looks at
 	 * the cfs_rq utilization to select a frequency.
@@ -6759,8 +6843,11 @@ enqueue_task_fair(struct rq *rq, struct
 		cpufreq_update_util(rq, SCHED_CPUFREQ_IOWAIT);
 
 	for_each_sched_entity(se) {
-		if (se->on_rq)
+		if (se->on_rq) {
+			if (se->sched_delayed)
+				requeue_delayed_entity(se);
 			break;
+		}
 		cfs_rq = cfs_rq_of(se);
 		enqueue_entity(cfs_rq, se, flags);
 
@@ -6836,6 +6923,7 @@ static int dequeue_entities(struct rq *r
 {
 	bool was_sched_idle = sched_idle_rq(rq);
 	bool task_sleep = flags & DEQUEUE_SLEEP;
+	bool task_delayed = flags & DEQUEUE_DELAYED;
 	struct task_struct *p = NULL;
 	struct cfs_rq *cfs_rq;
 	int idle_h_nr_running;
@@ -6849,7 +6937,13 @@ static int dequeue_entities(struct rq *r
 
 	for_each_sched_entity(se) {
 		cfs_rq = cfs_rq_of(se);
-		dequeue_entity(cfs_rq, se, flags);
+
+		if (!dequeue_entity(cfs_rq, se, flags)) {
+			if (p && &p->se == se)
+				return -1;
+
+			break;
+		}
 
 		/* h_nr_running is the hierachical count of tasks */
 		if (p) {
@@ -6877,6 +6971,7 @@ static int dequeue_entities(struct rq *r
 			break;
 		}
 		flags |= DEQUEUE_SLEEP;
+		flags &= ~DEQUEUE_DELAYED;
 	}
 
 	for_each_sched_entity(se) {
@@ -6906,6 +7001,18 @@ static int dequeue_entities(struct rq *r
 		/* balance early to pull high priority tasks */
 		if (unlikely(!was_sched_idle && sched_idle_rq(rq)))
 			rq->next_balance = jiffies;
+
+		if (task_delayed) {
+			SCHED_WARN_ON(!task_sleep);
+			SCHED_WARN_ON(p->on_rq != 1);
+
+			/* Fix-up what dequeue_task_fair() skipped */
+			util_est_update(&rq->cfs, p, task_sleep);
+			hrtick_update(rq);
+
+			/* Fix-up what deactivate_task() skipped. */
+			WRITE_ONCE(p->on_rq, 0);
+		}
 	}
 
 	return 1;
@@ -6923,6 +7030,10 @@ static bool dequeue_task_fair(struct rq
 	if (dequeue_entities(rq, &p->se, flags) < 0)
 		return false;
 
+	/*
+	 * It doesn't make sense to update util_est for the delayed dequeue
+	 * case where ttwu will make it appear the sleep never happened.
+	 */
 	util_est_update(&rq->cfs, p, flags & DEQUEUE_SLEEP);
 	hrtick_update(rq);
 	return true;
@@ -8463,7 +8574,9 @@ static struct task_struct *pick_task_fai
 		if (unlikely(check_cfs_rq_runtime(cfs_rq)))
 			goto again;
 
-		se = pick_next_entity(cfs_rq);
+		se = pick_next_entity(rq, cfs_rq);
+		if (!se)
+			goto again;
 		cfs_rq = group_cfs_rq(se);
 	} while (cfs_rq);
 
@@ -12803,10 +12916,17 @@ static void attach_task_cfs_rq(struct ta
 static void switched_from_fair(struct rq *rq, struct task_struct *p)
 {
 	detach_task_cfs_rq(p);
+	// XXX think harder on this
+	// this could actually be handled correctly I suppose; keep the whole
+	// se enqueued while boosted. horrible complexity though
+	p->se.sched_delayed = 0;
+	// XXX also vlag ?!?
 }
 
 static void switched_to_fair(struct rq *rq, struct task_struct *p)
 {
+	SCHED_WARN_ON(p->se.sched_delayed);
+
 	attach_task_cfs_rq(p);
 
 	set_task_max_allowed_capacity(p);
kernel/sched/features.h +12 -0
@@ -29,6 +29,18 @@ SCHED_FEAT(NEXT_BUDDY, false)
 SCHED_FEAT(CACHE_HOT_BUDDY, true)
 
 /*
+ * Delay dequeueing tasks until they get selected or woken.
+ *
+ * By delaying the dequeue for non-eligible tasks, they remain in the
+ * competition and can burn off their negative lag. When they get selected
+ * they'll have positive lag by definition.
+ *
+ * DELAY_ZERO clips the lag on dequeue (or wakeup) to 0.
+ */
+SCHED_FEAT(DELAY_DEQUEUE, true)
+SCHED_FEAT(DELAY_ZERO, true)
+
+/*
  * Allow wakeup-time preemption of the current task:
  */
 SCHED_FEAT(WAKEUP_PREEMPTION, true)
kernel/sched/sched.h +2 -0
@@ -2245,6 +2245,7 @@ extern const u32		sched_prio_to_wmult[40
 #define DEQUEUE_MOVE		0x04 /* Matches ENQUEUE_MOVE */
 #define DEQUEUE_NOCLOCK		0x08 /* Matches ENQUEUE_NOCLOCK */
 #define DEQUEUE_MIGRATING	0x100 /* Matches ENQUEUE_MIGRATING */
+#define DEQUEUE_DELAYED		0x200 /* Matches ENQUEUE_DELAYED */
 
 #define ENQUEUE_WAKEUP		0x01
 #define ENQUEUE_RESTORE		0x02
@@ -2260,6 +2261,7 @@ extern const u32		sched_prio_to_wmult[40
 #endif
 #define ENQUEUE_INITIAL		0x80
 #define ENQUEUE_MIGRATING	0x100
+#define ENQUEUE_DELAYED		0x200
 
 #define RETRY_TASK		((void *)-1UL)
 
#30 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Hi,

Quoted 115 lines · Luis Machado

Hi Peter,

Quoted 19 lines · Peter Zijlstra

Extend / fix 86bfbb7ce4f6 ("sched/fair: Add lag based placement") by noting that lag is fundamentally a temporal measure. It should not be carried around indefinitely.

OTOH it should also not be instantly discarded, doing so will allow a task to game the system by purposefully (micro) sleeping at the end of its time quantum.

Since lag is intimately tied to the virtual time base, a wall-time based decay is also insufficient, notably competition is required for any of this to make sense.

Instead, delay the dequeue and keep the 'tasks' on the runqueue, competing until they are eligible.

Strictly speaking, we only care about keeping them until the 0-lag point, but that is a difficult proposition, instead carry them around until they get picked again, and dequeue them at that point.

Quoted 6 lines

Since we should have dequeued them at the 0-lag point, truncate lag (eg. don't let them earn positive lag).

XXX test the cfs-throttle stuff

Playing around with a Pixel 6 running a 6.6-based kernel with this series backported on top, I spotted a noticeable performance improvement on the speedometer benchmark:

- m6.6-stock-* is the 6.6 mainline Android kernel unmodified.

- m6.6-eevdf-complete-* is the 6.6 mainline Android kernel with this series applied on top (along with a few required backported patches).

+-------------------+-----------------------+-----------+
|      metric       |          tag          | perc_diff |
+-------------------+-----------------------+-----------+
| Speedometer Score |     m6.6-stock-1      |   0.0%    |
| Speedometer Score |     m6.6-stock-2      |   1.23%   |
| Speedometer Score |     m6.6-stock-3      |  -0.22%   |
| Speedometer Score | m6.6-eevdf-complete-1 |   4.54%   |
| Speedometer Score | m6.6-eevdf-complete-2 |   4.95%   |
| Speedometer Score | m6.6-eevdf-complete-3 |   6.07%   |
+-------------------+-----------------------+-----------+

Also some interesting improvements in terms of frame timing for the uibenchjanktests benchmark. In particular the metrics of missed deadlines and jank (late) frames, which seems to indicate better latencies.

+-----------------------+-----------------------+-----------+
|        metric         |          tag          | perc_diff |
+-----------------------+-----------------------+-----------+
| gfx-avg-frame-time-50 |     m6.6-stock-1      |    0.0    |
| gfx-avg-frame-time-90 |     m6.6-stock-1      |    0.0    |
| gfx-avg-frame-time-95 |     m6.6-stock-1      |    0.0    |
| gfx-avg-frame-time-99 |     m6.6-stock-1      |    0.0    |
| gfx-avg-frame-time-50 |     m6.6-stock-2      |   3.46    |
| gfx-avg-frame-time-90 |     m6.6-stock-2      |   1.19    |
| gfx-avg-frame-time-95 |     m6.6-stock-2      |   0.24    |
| gfx-avg-frame-time-99 |     m6.6-stock-2      |   0.48    |
| gfx-avg-frame-time-50 | m6.6-eevdf-complete-1 |  -30.45   |
| gfx-avg-frame-time-90 | m6.6-eevdf-complete-1 |  -48.44   |
| gfx-avg-frame-time-95 | m6.6-eevdf-complete-1 |  -51.32   |
| gfx-avg-frame-time-99 | m6.6-eevdf-complete-1 |  -52.48   |
| gfx-avg-frame-time-50 | m6.6-eevdf-complete-2 |  -30.32   |
| gfx-avg-frame-time-90 | m6.6-eevdf-complete-2 |  -48.16   |
| gfx-avg-frame-time-95 | m6.6-eevdf-complete-2 |  -51.08   |
| gfx-avg-frame-time-99 | m6.6-eevdf-complete-2 |   -51.7   |
+-----------------------+-----------------------+-----------+
+-----------------------------------+-----------------------+-----------+
|              metric               |          tag          | perc_diff |
+-----------------------------------+-----------------------+-----------+
| gfx-avg-num-frame-deadline-missed |     m6.6-stock-1      |    0.0    |
| gfx-max-num-frame-deadline-missed |     m6.6-stock-1      |    0.0    |
| gfx-avg-num-frame-deadline-missed |     m6.6-stock-2      |   -3.21   |
| gfx-max-num-frame-deadline-missed |     m6.6-stock-2      |   -3.21   |
| gfx-avg-num-frame-deadline-missed | m6.6-eevdf-complete-1 |  -85.29   |
| gfx-max-num-frame-deadline-missed | m6.6-eevdf-complete-1 |  -85.29   |
| gfx-avg-num-frame-deadline-missed | m6.6-eevdf-complete-2 |   -84.8   |
| gfx-max-num-frame-deadline-missed | m6.6-eevdf-complete-2 |   -84.8   |
+-----------------------------------+-----------------------+-----------+
+----------------------------+-----------------------+-----------+
|           metric           |          tag          | perc_diff |
+----------------------------+-----------------------+-----------+
| gfx-avg-high-input-latency |     m6.6-stock-1      |    0.0    |
| gfx-max-high-input-latency |     m6.6-stock-1      |    0.0    |
| gfx-avg-high-input-latency |     m6.6-stock-2      |   0.93    |
| gfx-max-high-input-latency |     m6.6-stock-2      |   0.93    |
| gfx-avg-high-input-latency | m6.6-eevdf-complete-1 |  -18.35   |
| gfx-max-high-input-latency | m6.6-eevdf-complete-1 |  -18.35   |
| gfx-avg-high-input-latency | m6.6-eevdf-complete-2 |  -18.05   |
| gfx-max-high-input-latency | m6.6-eevdf-complete-2 |  -18.05   |
+----------------------------+-----------------------+-----------+
+--------------+-----------------------+-----------+
|    metric    |          tag          | perc_diff |
+--------------+-----------------------+-----------+
| gfx-avg-jank |     m6.6-stock-1      |    0.0    |
| gfx-max-jank |     m6.6-stock-1      |    0.0    |
| gfx-avg-jank |     m6.6-stock-2      |   1.56    |
| gfx-max-jank |     m6.6-stock-2      |   1.56    |
| gfx-avg-jank | m6.6-eevdf-complete-1 |  -82.81   |
| gfx-max-jank | m6.6-eevdf-complete-1 |  -82.81   |
| gfx-avg-jank | m6.6-eevdf-complete-2 |  -78.12   |
| gfx-max-jank | m6.6-eevdf-complete-2 |  -78.12   |
+--------------+-----------------------+-----------+
Quoted 6 lines

Bisecting through the patches in this series, I ended up with patch 08/10 as the one that improved things overall for these benchmarks.

I'd like to investigate this further to understand the reason behind some of these dramatic improvements.

Investigating these improvements a bit more, I noticed they came with a significantly higher power usage on the Pixel6 (where EAS is enabled). I bisected it down to the delayed dequeue patch. Disabling DELAY_DEQUEUE and DELAY_ZERO at runtime doesn't help in bringing the power usage down.

Though I don't fully understand the reason behind this change in behavior yet, I did spot the benchmark processes running almost entirely on the big core cluster, with little to no use of the little core and mid core clusters.

That would explain higher power usage and also the significant jump in performance.

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

author maintainer
#31 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 11 lines · Luis Machado
Quoted 6 lines

Bisecting through the patches in this series, I ended up with patch 08/10 as the one that improved things overall for these benchmarks.

I'd like to investigate this further to understand the reason behind some of these dramatic improvements.

Investigating these improvements a bit more, I noticed they came with a significantly higher power usage on the Pixel6 (where EAS is enabled). I bisected it down to the delayed dequeue patch. Disabling DELAY_DEQUEUE and DELAY_ZERO at runtime doesn't help in bringing the power usage down.

Hmm, that is unexpected. The intent was for NO_DELAY_DEQUEUE to fully disable things. I'll go have a prod at it.

Quoted 5 lines

Though I don't fully understand the reason behind this change in behavior yet, I did spot the benchmark processes running almost entirely on the big core cluster, with little to no use of the little core and mid core clusters.

That would explain higher power usage and also the significant jump in performance.

ISTR you (arm) has these tools to trace and plot the varioud util values. This should be readily reflected there if that is the case, no?

Quoted 3 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

I'll go over all that again.

author maintainer
#32 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

The PELT runnable values will be inflated because of delayed dequeue. cpu_util() uses those in the @boost case, and as such this can indeed affect things.

This can also slightly affect the cgroup case, but since the delay goes away as contention goes away, and the cgroup case must already assume worst case overlap, this seems limited.

/me goes ponder things moar.

author maintainer
#33 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 14 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

The PELT runnable values will be inflated because of delayed dequeue. cpu_util() uses those in the @boost case, and as such this can indeed affect things.

Quoted 5 lines

This can also slightly affect the cgroup case, but since the delay goes away as contention goes away, and the cgroup case must already assume worst case overlap, this seems limited.

/me goes ponder things moar.

First order approximation of a fix would be something like the totally untested below I suppose...

---

kernel/sched/fair.c +2 -0
index cfd1fd188d29..f3f70b5adca0 100644
@@ -5391,6 +5391,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;
 			se->sched_delayed = 1;
+			update_load_avg(cfs_rq, se, 0);
 			return false;
 		}
 	}
@@ -6817,6 +6818,7 @@ requeue_delayed_entity(struct sched_entity *se)
 	}
 
 	se->sched_delayed = 0;
+	update_load_avg(qcfs_rq, se, 0);
 }
 
 /*
kernel/sched/sched.h +6 -0
index d07a3b98f1fb..d16529613123 100644
@@ -810,6 +810,9 @@ static inline void se_update_runnable(struct sched_entity *se)
 
 static inline long se_runnable(struct sched_entity *se)
 {
+	if (se->sched_delayed)
+		return false;
+
 	if (entity_is_task(se))
 		return !!se->on_rq;
 	else
@@ -823,6 +826,9 @@ static inline void se_update_runnable(struct sched_entity *se) {}
 
 static inline long se_runnable(struct sched_entity *se)
 {
+	if (se->sched_delayed)
+		return false;
+
 	return !!se->on_rq;
 }
 #endif
author maintainer
#34 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 38 lines · Peter Zijlstra
Quoted 14 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

The PELT runnable values will be inflated because of delayed dequeue. cpu_util() uses those in the @boost case, and as such this can indeed affect things.

Quoted 5 lines

This can also slightly affect the cgroup case, but since the delay goes away as contention goes away, and the cgroup case must already assume worst case overlap, this seems limited.

/me goes ponder things moar.

First order approximation of a fix would be something like the totally untested below I suppose...

---

kernel/sched/fair.c +1 -0
index cfd1fd188d29..f3f70b5adca0 100644
@@ -5391,6 +5391,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;
 			se->sched_delayed = 1;
+			update_load_avg(cfs_rq, se, 0);
 			return false;
 		}
 	}

kernel/sched/fair.c requeue_delayed_entity(struct sched_entity *se) · line 6821

Quoted 4 lines
kernel/sched/fair.c +1 -0
@@ -6817,6 +6818,7 @@ requeue_delayed_entity(struct sched_entity *se)
 	}
 
 	se->sched_delayed = 0;
+	update_load_avg(qcfs_rq, se, 0);

Compiler demands this ^^^^^ be cfs_rq instead ;-) brown_paper_bags--;

Quoted 27 lines
kernel/sched/fair.c +0 -0
 }
 
 /*
kernel/sched/sched.h +6 -0
index d07a3b98f1fb..d16529613123 100644
@@ -810,6 +810,9 @@ static inline void se_update_runnable(struct sched_entity *se)
 
 static inline long se_runnable(struct sched_entity *se)
 {
+	if (se->sched_delayed)
+		return false;
+
 	if (entity_is_task(se))
 		return !!se->on_rq;
 	else
@@ -823,6 +826,9 @@ static inline void se_update_runnable(struct sched_entity *se) {}
 
 static inline long se_runnable(struct sched_entity *se)
 {
+	if (se->sched_delayed)
+		return false;
+
 	return !!se->on_rq;
 }
 #endif
#35 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra
Quoted 15 lines · Peter Zijlstra
Quoted 14 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

The PELT runnable values will be inflated because of delayed dequeue. cpu_util() uses those in the @boost case, and as such this can indeed affect things.

Quoted 8 lines
Quoted 5 lines

This can also slightly affect the cgroup case, but since the delay goes away as contention goes away, and the cgroup case must already assume worst case overlap, this seems limited.

/me goes ponder things moar.

First order approximation of a fix would be something like the totally untested below I suppose...

Thanks Peter. Let me give it a try and I'll report back.

Quoted 46 lines

---

kernel/sched/fair.c +2 -0
index cfd1fd188d29..f3f70b5adca0 100644
@@ -5391,6 +5391,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
 			if (cfs_rq->next == se)
 				cfs_rq->next = NULL;
 			se->sched_delayed = 1;
+			update_load_avg(cfs_rq, se, 0);
 			return false;
 		}
 	}
@@ -6817,6 +6818,7 @@ requeue_delayed_entity(struct sched_entity *se)
 	}
 
 	se->sched_delayed = 0;
+	update_load_avg(qcfs_rq, se, 0);
 }
 
 /*
kernel/sched/sched.h +6 -0
index d07a3b98f1fb..d16529613123 100644
@@ -810,6 +810,9 @@ static inline void se_update_runnable(struct sched_entity *se)
 
 static inline long se_runnable(struct sched_entity *se)
 {
+	if (se->sched_delayed)
+		return false;
+
 	if (entity_is_task(se))
 		return !!se->on_rq;
 	else
@@ -823,6 +826,9 @@ static inline void se_update_runnable(struct sched_entity *se) {}
 
 static inline long se_runnable(struct sched_entity *se)
 {
+	if (se->sched_delayed)
+		return false;
+
 	return !!se->on_rq;
 }
 #endif
#36 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra

Hi Peter,

Quoted 15 lines · Peter Zijlstra
Quoted 14 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

The PELT runnable values will be inflated because of delayed dequeue. cpu_util() uses those in the @boost case, and as such this can indeed affect things.

Quoted 8 lines
Quoted 5 lines

This can also slightly affect the cgroup case, but since the delay goes away as contention goes away, and the cgroup case must already assume worst case overlap, this seems limited.

/me goes ponder things moar.

First order approximation of a fix would be something like the totally untested below I suppose...

I gave this a try on the Pixel 6, and I noticed some improvement (see below), but not enough to bring it back to the original levels.

(1) m6.6-stock - Basic EEVDF with wakeup preemption fix (baseline)
(2) m6.6-eevdf-complete: m6.6-stock plus this series.
(3) m6.6-eevdf-complete-no-delay-dequeue: (2) + NO_DELAY_DEQUEUE
(4) m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero: (2) + NO_DELAY_DEQUEUE + NO_DELAY_ZERO
(5) m6.6-eevdf-complete-no-delay-zero: (2) + NO_DELAY_ZERO
(6) m6.6-eevdf-complete-pelt-fix: (2) + the proposed load_avg update patch.

I included (3), (4) and (5) to exercise the impact of disabling the individual scheduler features.

Energy use.

+------------+------------------------------------------------------+-----------+
|  cluster   |                         tag                          | perc_diff |
+------------+------------------------------------------------------+-----------+
|    CPU     |                   m6.6-stock                         |   0.0%    |
|  CPU-Big   |                   m6.6-stock                         |   0.0%    |
| CPU-Little |                   m6.6-stock                         |   0.0%    |
|  CPU-Mid   |                   m6.6-stock                         |   0.0%    |
|    GPU     |                   m6.6-stock                         |   0.0%    |
|   Total    |                   m6.6-stock                         |   0.0%    |
|    CPU     |                m6.6-eevdf-complete                   |  114.51%  |
|  CPU-Big   |                m6.6-eevdf-complete                   |  90.75%   |
| CPU-Little |                m6.6-eevdf-complete                   |  98.74%   |
|  CPU-Mid   |                m6.6-eevdf-complete                   |  213.9%   |
|    GPU     |                m6.6-eevdf-complete                   |  -7.04%   |
|   Total    |                m6.6-eevdf-complete                   |  100.92%  |
|    CPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  117.77%  |
|  CPU-Big   |        m6.6-eevdf-complete-no-delay-dequeue          |  113.79%  |
| CPU-Little |        m6.6-eevdf-complete-no-delay-dequeue          |  97.47%   |
|  CPU-Mid   |        m6.6-eevdf-complete-no-delay-dequeue          |  189.0%   |
|    GPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  -6.74%   |
|   Total    |        m6.6-eevdf-complete-no-delay-dequeue          |  103.84%  |
|    CPU     | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  120.45%  |
|  CPU-Big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  113.65%  |
| CPU-Little | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  99.04%   |
|  CPU-Mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  201.14%  |
|    GPU     | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  -5.37%   |
|   Total    | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  106.38%  |
|    CPU     |         m6.6-eevdf-complete-no-delay-zero            |  119.05%  |
|  CPU-Big   |         m6.6-eevdf-complete-no-delay-zero            |  107.55%  |
| CPU-Little |         m6.6-eevdf-complete-no-delay-zero            |  98.66%   |
|  CPU-Mid   |         m6.6-eevdf-complete-no-delay-zero            |  206.58%  |
|    GPU     |         m6.6-eevdf-complete-no-delay-zero            |  -5.25%   |
|   Total    |         m6.6-eevdf-complete-no-delay-zero            |  105.14%  |
|    CPU     |            m6.6-eevdf-complete-pelt-fix              |  105.56%  |
|  CPU-Big   |            m6.6-eevdf-complete-pelt-fix              |  100.45%  |
| CPU-Little |            m6.6-eevdf-complete-pelt-fix              |   94.4%   |
|  CPU-Mid   |            m6.6-eevdf-complete-pelt-fix              |  150.94%  |
|    GPU     |            m6.6-eevdf-complete-pelt-fix              |  -3.96%   |
|   Total    |            m6.6-eevdf-complete-pelt-fix              |  93.31%   |
+------------+------------------------------------------------------+-----------+

Utilization and load levels.

+---------+------------------------------------------------------+----------+-----------+
| cluster |                         tag                          | variable | perc_diff |
+---------+------------------------------------------------------+----------+-----------+
| little  |                   m6.6-stock                         |   load   |   0.0%    |
| little  |                   m6.6-stock                         |   util   |   0.0%    |
| little  |                m6.6-eevdf-complete                   |   load   |  29.56%   |
| little  |                m6.6-eevdf-complete                   |   util   |   55.4%   |
| little  |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |  42.89%   |
| little  |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  69.47%   |
| little  | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  51.05%   |
| little  | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |  76.55%   |
| little  |         m6.6-eevdf-complete-no-delay-zero            |   load   |  34.51%   |
| little  |         m6.6-eevdf-complete-no-delay-zero            |   util   |  72.53%   |
| little  |            m6.6-eevdf-complete-pelt-fix              |   load   |  29.96%   |
| little  |            m6.6-eevdf-complete-pelt-fix              |   util   |  59.82%   |
|   mid   |                   m6.6-stock                         |   load   |   0.0%    |
|   mid   |                   m6.6-stock                         |   util   |   0.0%    |
|   mid   |                m6.6-eevdf-complete                   |   load   |  29.37%   |
|   mid   |                m6.6-eevdf-complete                   |   util   |  75.22%   |
|   mid   |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |   36.4%   |
|   mid   |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  80.28%   |
|   mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  30.35%   |
|   mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |   90.2%   |
|   mid   |         m6.6-eevdf-complete-no-delay-zero            |   load   |  37.83%   |
|   mid   |         m6.6-eevdf-complete-no-delay-zero            |   util   |  93.79%   |
|   mid   |            m6.6-eevdf-complete-pelt-fix              |   load   |  33.57%   |
|   mid   |            m6.6-eevdf-complete-pelt-fix              |   util   |  67.83%   |
|   big   |                   m6.6-stock                         |   load   |   0.0%    |
|   big   |                   m6.6-stock                         |   util   |   0.0%    |
|   big   |                m6.6-eevdf-complete                   |   load   |  97.39%   |
|   big   |                m6.6-eevdf-complete                   |   util   |  12.63%   |
|   big   |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |  139.69%  |
|   big   |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  22.58%   |
|   big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  125.36%  |
|   big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |  23.15%   |
|   big   |         m6.6-eevdf-complete-no-delay-zero            |   load   |  128.56%  |
|   big   |         m6.6-eevdf-complete-no-delay-zero            |   util   |  25.03%   |
|   big   |            m6.6-eevdf-complete-pelt-fix              |   load   |  130.73%  |
|   big   |            m6.6-eevdf-complete-pelt-fix              |   util   |  17.52%   |
+---------+------------------------------------------------------+----------+-----------+
#37 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 118 lines · Luis Machado

Hi Peter,

Quoted 15 lines · Peter Zijlstra
Quoted 14 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

The PELT runnable values will be inflated because of delayed dequeue. cpu_util() uses those in the @boost case, and as such this can indeed affect things.

Quoted 8 lines
Quoted 5 lines

This can also slightly affect the cgroup case, but since the delay goes away as contention goes away, and the cgroup case must already assume worst case overlap, this seems limited.

/me goes ponder things moar.

First order approximation of a fix would be something like the totally untested below I suppose...

I gave this a try on the Pixel 6, and I noticed some improvement (see below), but not enough to bring it back to the original levels.

(1) m6.6-stock - Basic EEVDF with wakeup preemption fix (baseline)
(2) m6.6-eevdf-complete: m6.6-stock plus this series.
(3) m6.6-eevdf-complete-no-delay-dequeue: (2) + NO_DELAY_DEQUEUE
(4) m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero: (2) + NO_DELAY_DEQUEUE + NO_DELAY_ZERO
(5) m6.6-eevdf-complete-no-delay-zero: (2) + NO_DELAY_ZERO
(6) m6.6-eevdf-complete-pelt-fix: (2) + the proposed load_avg update patch.

I included (3), (4) and (5) to exercise the impact of disabling the individual scheduler features.

Energy use.

+------------+------------------------------------------------------+-----------+
|  cluster   |                         tag                          | perc_diff |
+------------+------------------------------------------------------+-----------+
|    CPU     |                   m6.6-stock                         |   0.0%    |
|  CPU-Big   |                   m6.6-stock                         |   0.0%    |
| CPU-Little |                   m6.6-stock                         |   0.0%    |
|  CPU-Mid   |                   m6.6-stock                         |   0.0%    |
|    GPU     |                   m6.6-stock                         |   0.0%    |
|   Total    |                   m6.6-stock                         |   0.0%    |
|    CPU     |                m6.6-eevdf-complete                   |  114.51%  |
|  CPU-Big   |                m6.6-eevdf-complete                   |  90.75%   |
| CPU-Little |                m6.6-eevdf-complete                   |  98.74%   |
|  CPU-Mid   |                m6.6-eevdf-complete                   |  213.9%   |
|    GPU     |                m6.6-eevdf-complete                   |  -7.04%   |
|   Total    |                m6.6-eevdf-complete                   |  100.92%  |
|    CPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  117.77%  |
|  CPU-Big   |        m6.6-eevdf-complete-no-delay-dequeue          |  113.79%  |
| CPU-Little |        m6.6-eevdf-complete-no-delay-dequeue          |  97.47%   |
|  CPU-Mid   |        m6.6-eevdf-complete-no-delay-dequeue          |  189.0%   |
|    GPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  -6.74%   |
|   Total    |        m6.6-eevdf-complete-no-delay-dequeue          |  103.84%  |
|    CPU     | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  120.45%  |
|  CPU-Big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  113.65%  |
| CPU-Little | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  99.04%   |
|  CPU-Mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  201.14%  |
|    GPU     | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  -5.37%   |
|   Total    | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  106.38%  |
|    CPU     |         m6.6-eevdf-complete-no-delay-zero            |  119.05%  |
|  CPU-Big   |         m6.6-eevdf-complete-no-delay-zero            |  107.55%  |
| CPU-Little |         m6.6-eevdf-complete-no-delay-zero            |  98.66%   |
|  CPU-Mid   |         m6.6-eevdf-complete-no-delay-zero            |  206.58%  |
|    GPU     |         m6.6-eevdf-complete-no-delay-zero            |  -5.25%   |
|   Total    |         m6.6-eevdf-complete-no-delay-zero            |  105.14%  |
|    CPU     |            m6.6-eevdf-complete-pelt-fix              |  105.56%  |
|  CPU-Big   |            m6.6-eevdf-complete-pelt-fix              |  100.45%  |
| CPU-Little |            m6.6-eevdf-complete-pelt-fix              |   94.4%   |
|  CPU-Mid   |            m6.6-eevdf-complete-pelt-fix              |  150.94%  |
|    GPU     |            m6.6-eevdf-complete-pelt-fix              |  -3.96%   |
|   Total    |            m6.6-eevdf-complete-pelt-fix              |  93.31%   |
+------------+------------------------------------------------------+-----------+

Utilization and load levels.

+---------+------------------------------------------------------+----------+-----------+
| cluster |                         tag                          | variable | perc_diff |
+---------+------------------------------------------------------+----------+-----------+
| little  |                   m6.6-stock                         |   load   |   0.0%    |
| little  |                   m6.6-stock                         |   util   |   0.0%    |
| little  |                m6.6-eevdf-complete                   |   load   |  29.56%   |
| little  |                m6.6-eevdf-complete                   |   util   |   55.4%   |
| little  |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |  42.89%   |
| little  |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  69.47%   |
| little  | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  51.05%   |
| little  | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |  76.55%   |
| little  |         m6.6-eevdf-complete-no-delay-zero            |   load   |  34.51%   |
| little  |         m6.6-eevdf-complete-no-delay-zero            |   util   |  72.53%   |
| little  |            m6.6-eevdf-complete-pelt-fix              |   load   |  29.96%   |
| little  |            m6.6-eevdf-complete-pelt-fix              |   util   |  59.82%   |
|   mid   |                   m6.6-stock                         |   load   |   0.0%    |
|   mid   |                   m6.6-stock                         |   util   |   0.0%    |
|   mid   |                m6.6-eevdf-complete                   |   load   |  29.37%   |
|   mid   |                m6.6-eevdf-complete                   |   util   |  75.22%   |
|   mid   |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |   36.4%   |
|   mid   |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  80.28%   |
|   mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  30.35%   |
|   mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |   90.2%   |
|   mid   |         m6.6-eevdf-complete-no-delay-zero            |   load   |  37.83%   |
|   mid   |         m6.6-eevdf-complete-no-delay-zero            |   util   |  93.79%   |
|   mid   |            m6.6-eevdf-complete-pelt-fix              |   load   |  33.57%   |
|   mid   |            m6.6-eevdf-complete-pelt-fix              |   util   |  67.83%   |
|   big   |                   m6.6-stock                         |   load   |   0.0%    |
|   big   |                   m6.6-stock                         |   util   |   0.0%    |
|   big   |                m6.6-eevdf-complete                   |   load   |  97.39%   |
|   big   |                m6.6-eevdf-complete                   |   util   |  12.63%   |
|   big   |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |  139.69%  |
Quoted 8 lines
|   big   |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  22.58%   |
|   big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  125.36%  |
|   big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |  23.15%   |
|   big   |         m6.6-eevdf-complete-no-delay-zero            |   load   |  128.56%  |
|   big   |         m6.6-eevdf-complete-no-delay-zero            |   util   |  25.03%   |
|   big   |            m6.6-eevdf-complete-pelt-fix              |   load   |  130.73%  |
|   big   |            m6.6-eevdf-complete-pelt-fix              |   util   |  17.52%   |
+---------+------------------------------------------------------+----------+-----------+

Going through the code, my understanding is that the util_est functions seem to be getting called correctly, and in the right order. That is, we first util_est_enqueue, then util_est_dequeue and finally util_est_update. So the stats *should* be correct.

On dequeuing (dequeue_task_fair), we immediately call util_est_dequeue, even for the case of a DEQUEUE_DELAYED task, since we're no longer going to run the dequeue_delayed task for now, even though it is still in the rq.

We delay the util_est_update of dequeue_delayed tasks until a later time in dequeue_entities.

Eventually the dequeue_delayed task will have its lag zeroed when it becomes eligible again, (requeue_delayed_entity) while still being in the rq. It will then get dequeued/enqueued (requeued), and marked as a non-dequeue-delayed task.

Next time we attempt to enqueue such a task (enqueue_task_fair), it will skip the ENQUEUE_DELAYED block and call util_est_enqueue.

Still, something seems to be signalling that util/load is high, and causing migration to the big cores.

Maybe we're not decaying the util/load properly at some point, and inflated numbers start to happen.

I'll continue investigating.

#38 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 142 lines · Luis Machado
Quoted 118 lines · Luis Machado

Hi Peter,

Quoted 15 lines · Peter Zijlstra
Quoted 14 lines · Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

The PELT runnable values will be inflated because of delayed dequeue. cpu_util() uses those in the @boost case, and as such this can indeed affect things.

Quoted 8 lines
Quoted 5 lines

This can also slightly affect the cgroup case, but since the delay goes away as contention goes away, and the cgroup case must already assume worst case overlap, this seems limited.

/me goes ponder things moar.

First order approximation of a fix would be something like the totally untested below I suppose...

I gave this a try on the Pixel 6, and I noticed some improvement (see below), but not enough to bring it back to the original levels.

(1) m6.6-stock - Basic EEVDF with wakeup preemption fix (baseline)
(2) m6.6-eevdf-complete: m6.6-stock plus this series.
(3) m6.6-eevdf-complete-no-delay-dequeue: (2) + NO_DELAY_DEQUEUE
(4) m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero: (2) + NO_DELAY_DEQUEUE + NO_DELAY_ZERO
(5) m6.6-eevdf-complete-no-delay-zero: (2) + NO_DELAY_ZERO
(6) m6.6-eevdf-complete-pelt-fix: (2) + the proposed load_avg update patch.

I included (3), (4) and (5) to exercise the impact of disabling the individual scheduler features.

Energy use.

+------------+------------------------------------------------------+-----------+
|  cluster   |                         tag                          | perc_diff |
+------------+------------------------------------------------------+-----------+
|    CPU     |                   m6.6-stock                         |   0.0%    |
|  CPU-Big   |                   m6.6-stock                         |   0.0%    |
| CPU-Little |                   m6.6-stock                         |   0.0%    |
|  CPU-Mid   |                   m6.6-stock                         |   0.0%    |
|    GPU     |                   m6.6-stock                         |   0.0%    |
|   Total    |                   m6.6-stock                         |   0.0%    |
|    CPU     |                m6.6-eevdf-complete                   |  114.51%  |
|  CPU-Big   |                m6.6-eevdf-complete                   |  90.75%   |
| CPU-Little |                m6.6-eevdf-complete                   |  98.74%   |
|  CPU-Mid   |                m6.6-eevdf-complete                   |  213.9%   |
|    GPU     |                m6.6-eevdf-complete                   |  -7.04%   |
|   Total    |                m6.6-eevdf-complete                   |  100.92%  |
|    CPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  117.77%  |
|  CPU-Big   |        m6.6-eevdf-complete-no-delay-dequeue          |  113.79%  |
| CPU-Little |        m6.6-eevdf-complete-no-delay-dequeue          |  97.47%   |
|  CPU-Mid   |        m6.6-eevdf-complete-no-delay-dequeue          |  189.0%   |
|    GPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  -6.74%   |
|   Total    |        m6.6-eevdf-complete-no-delay-dequeue          |  103.84%  |
|    CPU     | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  120.45%  |
|  CPU-Big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  113.65%  |
| CPU-Little | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  99.04%   |
|  CPU-Mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  201.14%  |
|    GPU     | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  -5.37%   |
|   Total    | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |  106.38%  |
|    CPU     |         m6.6-eevdf-complete-no-delay-zero            |  119.05%  |
|  CPU-Big   |         m6.6-eevdf-complete-no-delay-zero            |  107.55%  |
| CPU-Little |         m6.6-eevdf-complete-no-delay-zero            |  98.66%   |
|  CPU-Mid   |         m6.6-eevdf-complete-no-delay-zero            |  206.58%  |
|    GPU     |         m6.6-eevdf-complete-no-delay-zero            |  -5.25%   |
|   Total    |         m6.6-eevdf-complete-no-delay-zero            |  105.14%  |
|    CPU     |            m6.6-eevdf-complete-pelt-fix              |  105.56%  |
|  CPU-Big   |            m6.6-eevdf-complete-pelt-fix              |  100.45%  |
| CPU-Little |            m6.6-eevdf-complete-pelt-fix              |   94.4%   |
|  CPU-Mid   |            m6.6-eevdf-complete-pelt-fix              |  150.94%  |
|    GPU     |            m6.6-eevdf-complete-pelt-fix              |  -3.96%   |
|   Total    |            m6.6-eevdf-complete-pelt-fix              |  93.31%   |
+------------+------------------------------------------------------+-----------+

Utilization and load levels.

+---------+------------------------------------------------------+----------+-----------+
| cluster |                         tag                          | variable | perc_diff |
+---------+------------------------------------------------------+----------+-----------+
| little  |                   m6.6-stock                         |   load   |   0.0%    |
| little  |                   m6.6-stock                         |   util   |   0.0%    |
| little  |                m6.6-eevdf-complete                   |   load   |  29.56%   |
| little  |                m6.6-eevdf-complete                   |   util   |   55.4%   |
| little  |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |  42.89%   |
| little  |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  69.47%   |
| little  | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  51.05%   |
| little  | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |  76.55%   |
| little  |         m6.6-eevdf-complete-no-delay-zero            |   load   |  34.51%   |
| little  |         m6.6-eevdf-complete-no-delay-zero            |   util   |  72.53%   |
| little  |            m6.6-eevdf-complete-pelt-fix              |   load   |  29.96%   |
| little  |            m6.6-eevdf-complete-pelt-fix              |   util   |  59.82%   |
|   mid   |                   m6.6-stock                         |   load   |   0.0%    |
|   mid   |                   m6.6-stock                         |   util   |   0.0%    |
|   mid   |                m6.6-eevdf-complete                   |   load   |  29.37%   |
|   mid   |                m6.6-eevdf-complete                   |   util   |  75.22%   |
|   mid   |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |   36.4%   |
|   mid   |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  80.28%   |
|   mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  30.35%   |
|   mid   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |   90.2%   |
|   mid   |         m6.6-eevdf-complete-no-delay-zero            |   load   |  37.83%   |
|   mid   |         m6.6-eevdf-complete-no-delay-zero            |   util   |  93.79%   |
|   mid   |            m6.6-eevdf-complete-pelt-fix              |   load   |  33.57%   |
|   mid   |            m6.6-eevdf-complete-pelt-fix              |   util   |  67.83%   |
|   big   |                   m6.6-stock                         |   load   |   0.0%    |
|   big   |                   m6.6-stock                         |   util   |   0.0%    |
|   big   |                m6.6-eevdf-complete                   |   load   |  97.39%   |
|   big   |                m6.6-eevdf-complete                   |   util   |  12.63%   |
|   big   |        m6.6-eevdf-complete-no-delay-dequeue          |   load   |  139.69%  |
Quoted 8 lines
|   big   |        m6.6-eevdf-complete-no-delay-dequeue          |   util   |  22.58%   |
|   big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   load   |  125.36%  |
|   big   | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero   |   util   |  23.15%   |
|   big   |         m6.6-eevdf-complete-no-delay-zero            |   load   |  128.56%  |
|   big   |         m6.6-eevdf-complete-no-delay-zero            |   util   |  25.03%   |
|   big   |            m6.6-eevdf-complete-pelt-fix              |   load   |  130.73%  |
|   big   |            m6.6-eevdf-complete-pelt-fix              |   util   |  17.52%   |
+---------+------------------------------------------------------+----------+-----------+

Going through the code, my understanding is that the util_est functions seem to be getting called correctly, and in the right order. That is, we first util_est_enqueue, then util_est_dequeue and finally util_est_update. So the stats *should* be correct.

On dequeuing (dequeue_task_fair), we immediately call util_est_dequeue, even for the case of a DEQUEUE_DELAYED task, since we're no longer going to run the dequeue_delayed task for now, even though it is still in the rq.

We delay the util_est_update of dequeue_delayed tasks until a later time in dequeue_entities.

Eventually the dequeue_delayed task will have its lag zeroed when it becomes eligible again, (requeue_delayed_entity) while still being in the rq. It will then get dequeued/enqueued (requeued), and marked as a non-dequeue-delayed task.

Quoted 9 lines

Next time we attempt to enqueue such a task (enqueue_task_fair), it will skip the ENQUEUE_DELAYED block and call util_est_enqueue.

Still, something seems to be signalling that util/load is high, and causing migration to the big cores.

Maybe we're not decaying the util/load properly at some point, and inflated numbers start to happen.

I'll continue investigating.

Just a quick update on this. While investigating this behavior, I spotted very high loadavg values on an idle system. For instance:

load average: 4733.84, 4721.24, 4680.33

I wonder if someone else also spotted this.

These values keep increasing slowly but steadily. When the system is under load they increase a bit more rapidly. Makes me wonder if we're missing decrementing nr_uninterruptible in some path, since that is what seems to be causing the loadavg to be off.

author maintainer
#39 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 6 lines · Luis Machado

Just a quick update on this. While investigating this behavior, I spotted very high loadavg values on an idle system. For instance:

load average: 4733.84, 4721.24, 4680.33

I wonder if someone else also spotted this.

Hadn't spotted it, but now that you mention it, I can definitely see it.

Let me go prod with something sharp. Thanks!

author maintainer
#40 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · Luis Machado

Just a quick update on this. While investigating this behavior, I spotted very high loadavg values on an idle system. For instance:

load average: 4733.84, 4721.24, 4680.33

I wonder if someone else also spotted this.

Hadn't spotted it, but now that you mention it, I can definitely see it.

Let me go prod with something sharp. Thanks!

What's the point of making notes if you then don't read them... *sigh*.

Does this help?

---
 kernel/sched/core.c  | 23 ++++++++++++-----------
 kernel/sched/fair.c  |  4 ++--
 kernel/sched/sched.h |  8 ++++++++
 3 files changed, 22 insertions(+), 13 deletions(-)
kernel/sched/core.c +12 -11
index 5ffd7e047393..43f061bcfe54 100644
@@ -2154,14 +2154,18 @@ void activate_task(struct rq *rq, struct task_struct *p, int flags)
 
 void deactivate_task(struct rq *rq, struct task_struct *p, int flags)
 {
-	bool sleep = flags & DEQUEUE_SLEEP;
+	SCHED_WARN_ON(flags & DEQUEUE_SLEEP);
 
-	if (dequeue_task(rq, p, flags)) {
-		WRITE_ONCE(p->on_rq, sleep ? 0 : TASK_ON_RQ_MIGRATING);
-		ASSERT_EXCLUSIVE_WRITER(p->on_rq);
-	} else {
-		SCHED_WARN_ON(!sleep); /* only sleep can fail */
-	}
+	dequeue_task(rq, p, flags);
+
+	WRITE_ONCE(p->on_rq, TASK_ON_RQ_MIGRATING);
+	ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+}
+
+static void block_task(struct rq *rq, struct task_struct *p, int flags)
+{
+	if (dequeue_task(rq, p, DEQUEUE_SLEEP | flags))
+		__block_task(rq, p);
 }
 
 static inline int __normal_prio(int policy, int rt_prio, int nice)
@@ -6693,9 +6697,6 @@ static void __sched notrace __schedule(unsigned int sched_mode)
 				!(prev_state & TASK_NOLOAD) &&
 				!(prev_state & TASK_FROZEN);
 
-			if (prev->sched_contributes_to_load)
-				rq->nr_uninterruptible++;
-
 			/*
 			 * __schedule()			ttwu()
 			 *   prev_state = prev->state;    if (p->on_rq && ...)
@@ -6707,7 +6708,7 @@ static void __sched notrace __schedule(unsigned int sched_mode)
 			 *
 			 * After this, schedule() must not care about p->state any more.
 			 */
-			deactivate_task(rq, prev, DEQUEUE_SLEEP | DEQUEUE_NOCLOCK);
+			block_task(rq, prev, DEQUEUE_SLEEP | DEQUEUE_NOCLOCK);
 
 			if (prev->in_iowait) {
 				atomic_inc(&rq->nr_iowait);
kernel/sched/fair.c +2 -2
index 536eabcb1a71..596a5fabe490 100644
@@ -7032,8 +7032,8 @@ static int dequeue_entities(struct rq *rq, struct sched_entity *se, int flags)
 			util_est_update(&rq->cfs, p, task_sleep);
 			hrtick_update(rq);
 
-			/* Fix-up what deactivate_task() skipped. */
-			WRITE_ONCE(p->on_rq, 0);
+			/* Fix-up what block_task() skipped. */
+			__block_task(rq, p);
 		}
 	}
 
kernel/sched/sched.h +8 -0
index 12841d8195c5..48e5f49d9bc2 100644
@@ -2560,6 +2560,14 @@ static inline void sub_nr_running(struct rq *rq, unsigned count)
 	sched_update_tick_dependency(rq);
 }
 
+static inline void __block_task(struct rq *rq, struct task_struct *p)
+{
+	WRITE_ONCE(p->on_rq, 0);
+	ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+	if (p->sched_contributes_to_load)
+		rq->nr_uninterruptible++;
+}
+
 extern void activate_task(struct rq *rq, struct task_struct *p, int flags);
 extern void deactivate_task(struct rq *rq, struct task_struct *p, int flags);
 
#41 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 10 lines · Peter Zijlstra
Quoted 9 lines · Peter Zijlstra
Quoted 6 lines · Luis Machado

Just a quick update on this. While investigating this behavior, I spotted very high loadavg values on an idle system. For instance:

load average: 4733.84, 4721.24, 4680.33

I wonder if someone else also spotted this.

Hadn't spotted it, but now that you mention it, I can definitely see it.

(ditto.. wart not enormous enough for awesome powers of observation:)

Quoted 6 lines
Quoted 2 lines

Let me go prod with something sharp. Thanks!

What's the point of making notes if you then don't read them... *sigh*.

Does this help?

That fixed it up here.

Quoted 89 lines

---
 kernel/sched/core.c  | 23 ++++++++++++----------- kernel/sched/fair.c  |  4 ++-- kernel/sched/sched.h |  8 ++++++++

 3 files changed, 22 insertions(+), 13 deletions(-)
kernel/sched/core.c +0 -0
index 5ffd7e047393..43f061bcfe54 100644
@@ -2154,14 +2154,18 @@ void activate_task(struct rq *rq, struct task_struct *p, int flags)


 void deactivate_task(struct rq *rq, struct task_struct *p, int flags) {
-       bool sleep = flags & DEQUEUE_SLEEP;
+       SCHED_WARN_ON(flags & DEQUEUE_SLEEP);

-       if (dequeue_task(rq, p, flags)) {
-               WRITE_ONCE(p->on_rq, sleep ? 0 : TASK_ON_RQ_MIGRATING); -               ASSERT_EXCLUSIVE_WRITER(p->on_rq);
-       } else {
-               SCHED_WARN_ON(!sleep); /* only sleep can fail */ -       }
+       dequeue_task(rq, p, flags);

kernel/sched/core.c +9 -0
+
+       WRITE_ONCE(p->on_rq, TASK_ON_RQ_MIGRATING);
+       ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+}
+
+static void block_task(struct rq *rq, struct task_struct *p, int flags)
+{
+       if (dequeue_task(rq, p, DEQUEUE_SLEEP | flags))
+               __block_task(rq, p);

 }

 static inline int __normal_prio(int policy, int rt_prio, int nice)

kernel/sched/core.c +0 -0
@@ -6693,9 +6697,6 @@ static void __sched notrace __schedule(unsigned int sched_mode)

                                !(prev_state & TASK_NOLOAD) && !(prev_state & TASK_FROZEN);

-                       if (prev->sched_contributes_to_load)
-                               rq->nr_uninterruptible++;
-
                        /*
                         * __schedule()                 ttwu()
                         *   prev_state = prev->state;    if (p->on_rq && ...)

kernel/sched/core.c +1 -1
@@ -6707,7 +6708,7 @@ static void __sched notrace __schedule(unsigned int sched_mode)
                          *
                          * After this, schedule() must not care about p->state any more.
                          */
-                       deactivate_task(rq, prev, DEQUEUE_SLEEP | DEQUEUE_NOCLOCK);
+                       block_task(rq, prev, DEQUEUE_SLEEP | DEQUEUE_NOCLOCK);


                        if (prev->in_iowait) { atomic_inc(&rq->nr_iowait);

kernel/sched/fair.c +0 -0
index 536eabcb1a71..596a5fabe490 100644
@@ -7032,8 +7032,8 @@ static int dequeue_entities(struct rq *rq, struct sched_entity *se, int flags)
                         util_est_update(&rq->cfs, p, task_sleep);
                         hrtick_update(rq);


-                       /* Fix-up what deactivate_task() skipped. */ -                       WRITE_ONCE(p->on_rq, 0);
+                       /* Fix-up what block_task() skipped. */ +                       __block_task(rq, p);
                }
        }

kernel/sched/sched.h +0 -0
index 12841d8195c5..48e5f49d9bc2 100644
@@ -2560,6 +2560,14 @@ static inline void sub_nr_running(struct rq *rq, unsigned count)
         sched_update_tick_dependency(rq);
  }

kernel/sched/sched.h +8 -0
+static inline void __block_task(struct rq *rq, struct task_struct *p)
+{
+       WRITE_ONCE(p->on_rq, 0);
+       ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+       if (p->sched_contributes_to_load)
+               rq->nr_uninterruptible++;
+}
+

 extern void activate_task(struct rq *rq, struct task_struct *p, int flags); extern void deactivate_task(struct rq *rq, struct task_struct *p, int flags);

#42 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra
Quoted 7 lines · Peter Zijlstra
Quoted 6 lines · Peter Zijlstra
Quoted 5 lines · Luis Machado

Just a quick update on this. While investigating this behavior, I spotted very high loadavg values on an idle system. For instance:

load average: 4733.84, 4721.24, 4680.33

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

I wonder if someone else also spotted this.

Hadn't spotted it, but now that you mention it, I can definitely see it.

Let me go prod with something sharp. Thanks!

What's the point of making notes if you then don't read them... *sigh*.

Makes it look like you did read them? :-)

Quoted 3 lines

Does this help?

It does address the load_avg issues as Mike G pointed out. Thanks for the quick patch!

Unfortunately, for me, it didn't help with the energy regression. So my investigation continues.

Something still seems to be driving things out of the smaller cores and into the bigger cores. And the load_avg accounting proved to be a red herring.

Quoted 89 lines
---
 kernel/sched/core.c  | 23 ++++++++++++-----------
 kernel/sched/fair.c  |  4 ++--
 kernel/sched/sched.h |  8 ++++++++
 3 files changed, 22 insertions(+), 13 deletions(-)
kernel/sched/core.c +12 -11
index 5ffd7e047393..43f061bcfe54 100644
@@ -2154,14 +2154,18 @@ void activate_task(struct rq *rq, struct task_struct *p, int flags)
 
 void deactivate_task(struct rq *rq, struct task_struct *p, int flags)
 {
-	bool sleep = flags & DEQUEUE_SLEEP;
+	SCHED_WARN_ON(flags & DEQUEUE_SLEEP);
 
-	if (dequeue_task(rq, p, flags)) {
-		WRITE_ONCE(p->on_rq, sleep ? 0 : TASK_ON_RQ_MIGRATING);
-		ASSERT_EXCLUSIVE_WRITER(p->on_rq);
-	} else {
-		SCHED_WARN_ON(!sleep); /* only sleep can fail */
-	}
+	dequeue_task(rq, p, flags);
+
+	WRITE_ONCE(p->on_rq, TASK_ON_RQ_MIGRATING);
+	ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+}
+
+static void block_task(struct rq *rq, struct task_struct *p, int flags)
+{
+	if (dequeue_task(rq, p, DEQUEUE_SLEEP | flags))
+		__block_task(rq, p);
 }
 
 static inline int __normal_prio(int policy, int rt_prio, int nice)
@@ -6693,9 +6697,6 @@ static void __sched notrace __schedule(unsigned int sched_mode)
 				!(prev_state & TASK_NOLOAD) &&
 				!(prev_state & TASK_FROZEN);
 
-			if (prev->sched_contributes_to_load)
-				rq->nr_uninterruptible++;
-
 			/*
 			 * __schedule()			ttwu()
 			 *   prev_state = prev->state;    if (p->on_rq && ...)
@@ -6707,7 +6708,7 @@ static void __sched notrace __schedule(unsigned int sched_mode)
 			 *
 			 * After this, schedule() must not care about p->state any more.
 			 */
-			deactivate_task(rq, prev, DEQUEUE_SLEEP | DEQUEUE_NOCLOCK);
+			block_task(rq, prev, DEQUEUE_SLEEP | DEQUEUE_NOCLOCK);
 
 			if (prev->in_iowait) {
 				atomic_inc(&rq->nr_iowait);
kernel/sched/fair.c +2 -2
index 536eabcb1a71..596a5fabe490 100644
@@ -7032,8 +7032,8 @@ static int dequeue_entities(struct rq *rq, struct sched_entity *se, int flags)
 			util_est_update(&rq->cfs, p, task_sleep);
 			hrtick_update(rq);
 
-			/* Fix-up what deactivate_task() skipped. */
-			WRITE_ONCE(p->on_rq, 0);
+			/* Fix-up what block_task() skipped. */
+			__block_task(rq, p);
 		}
 	}
 
kernel/sched/sched.h +8 -0
index 12841d8195c5..48e5f49d9bc2 100644
@@ -2560,6 +2560,14 @@ static inline void sub_nr_running(struct rq *rq, unsigned count)
 	sched_update_tick_dependency(rq);
 }
 
+static inline void __block_task(struct rq *rq, struct task_struct *p)
+{
+	WRITE_ONCE(p->on_rq, 0);
+	ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+	if (p->sched_contributes_to_load)
+		rq->nr_uninterruptible++;
+}
+
 extern void activate_task(struct rq *rq, struct task_struct *p, int flags);
 extern void deactivate_task(struct rq *rq, struct task_struct *p, int flags);
 
author maintainer
#43 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Luis Machado
Quoted 14 lines · Luis Machado

Going through the code, my understanding is that the util_est functions seem to be getting called correctly, and in the right order. That is, we first util_est_enqueue, then util_est_dequeue and finally util_est_update. So the stats *should* be correct.

On dequeuing (dequeue_task_fair), we immediately call util_est_dequeue, even for the case of a DEQUEUE_DELAYED task, since we're no longer going to run the dequeue_delayed task for now, even though it is still in the rq.

We delay the util_est_update of dequeue_delayed tasks until a later time in dequeue_entities.

Eventually the dequeue_delayed task will have its lag zeroed when it becomes eligible again, (requeue_delayed_entity) while still being in the rq. It will then get dequeued/enqueued (requeued), and marked as a non-dequeue-delayed task.

Quoted 8 lines

Next time we attempt to enqueue such a task (enqueue_task_fair), it will skip the ENQUEUE_DELAYED block and call util_est_enqueue.

Still, something seems to be signalling that util/load is high, and causing migration to the big cores.

Maybe we're not decaying the util/load properly at some point, and inflated numbers start to happen.

I'll continue investigating.

So I re-read all this util_est stuff again this evening and I am confused :-) Please bear with me.

So the old code does:

	dequeue_task_fair()
	  util_est_dequeue();
	  // actual dequeue -- updates pelt
	  util_est_update();

While the new code does:

	dequeue_task_fair()
	  util_est_dequeue();
	  if (!dequeue())
	    return;
	  util_est_update();

	delayed_dequeue:
	  util_est_update();

Specifically, we only call util_est_update() if/when we do the actual dequeue -- potentially at a later point in time. Because, as I argued in the comments, ttwu()'s ENQUEUE_DELAYED will not actually enqueue the task (since it is still enqueued) and therefore the update would be spurious.

However!!, if we do dequeue, then we'll end up updating the EWMA with a potentially different task_util(p).

And this is where the confusion comes... this extra time on the runqueue would not be running and thus decreate util_avg, as such task_util_est() should be lower than before and tasks should tend towards smaller cores, rather than larger cores as you seem to be seeing.

[ there is the 'dequeued + UTIL_EST_MARGIN < task_runnable()' escape
clause, which we might be hitting... dunno ]

In any case, if you haven't tried it already, could I ask you to test the below (once you're back in the office)?

Also, this doesn't really explain why things go sideways once you enable DELAY_DEQUEUE and then don't re-align if you disable it again. I mean, it should eventually age out the dodgy EWMA contributions and start working as expected.

kernel/sched/fair.c +3 -6
@@ -7008,7 +7008,6 @@ static int dequeue_entities(struct rq *r
 			SCHED_WARN_ON(p->on_rq != 1);
 
 			/* Fix-up what dequeue_task_fair() skipped */
-			util_est_update(&rq->cfs, p, task_sleep);
 			hrtick_update(rq);
 
 			/* Fix-up what block_task() skipped. */
@@ -7028,13 +7027,11 @@ static bool dequeue_task_fair(struct rq
 {
 	util_est_dequeue(&rq->cfs, p);
 
-	if (dequeue_entities(rq, &p->se, flags) < 0)
+	if (dequeue_entities(rq, &p->se, flags) < 0) {
+		util_est_update(&rq->cfs, p, DEQUEUE_SLEEP);
 		return false;
+	}
 
-	/*
-	 * It doesn't make sense to update util_est for the delayed dequeue
-	 * case where ttwu will make it appear the sleep never happened.
-	 */
 	util_est_update(&rq->cfs, p, flags & DEQUEUE_SLEEP);
 	hrtick_update(rq);
 	return true;
#44 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 16 lines · Peter Zijlstra
Quoted 14 lines · Luis Machado

Going through the code, my understanding is that the util_est functions seem to be getting called correctly, and in the right order. That is, we first util_est_enqueue, then util_est_dequeue and finally util_est_update. So the stats *should* be correct.

On dequeuing (dequeue_task_fair), we immediately call util_est_dequeue, even for the case of a DEQUEUE_DELAYED task, since we're no longer going to run the dequeue_delayed task for now, even though it is still in the rq.

We delay the util_est_update of dequeue_delayed tasks until a later time in dequeue_entities.

Eventually the dequeue_delayed task will have its lag zeroed when it becomes eligible again, (requeue_delayed_entity) while still being in the rq. It will then get dequeued/enqueued (requeued), and marked as a non-dequeue-delayed task.

Quoted 8 lines
Quoted 8 lines

Next time we attempt to enqueue such a task (enqueue_task_fair), it will skip the ENQUEUE_DELAYED block and call util_est_enqueue.

Still, something seems to be signalling that util/load is high, and causing migration to the big cores.

Maybe we're not decaying the util/load properly at some point, and inflated numbers start to happen.

I'll continue investigating.

Apologies for not getting this earlier, but I wanted to collect as much data as possible so we can hopefully see what's going wrong with this particular patch in the series.

(picking up from the previous questions on find_energy_efficient_cpu and overutilized)

For the rate of calls to find_energy_efficient_cpu, I checked again and I didn't see any significant differences there. With delayed dequeue or without delayed dequeue, I see almost the same number of calls, with the expected small variation between runs.

As for the overutilized part, there is a bit of a change within find_energy_efficient_cpu. When I first enable delayed dequeue via the debugfs (I boot up the system with it disabled for a clean state), I start to see a lot more cases of us bailing out of find_energy_efficient_cpu due to the overutilized condition.

Strangely though, things seem to stabilize after a while, and the rate of overutilized conditions when within find_energy_efficient_cpu goes down to normal levels.

Here's some numbers from a trace session:

Delayed dequeue off (clean state, 2 iterations):
------------------------------------------------

Calls to find_energy_efficient_cpu: 211087 Overutilized when in find_energy_efficient_cpu: 36224

Calls to find_energy_efficient_cpu: 205804 Overutilized when in find_energy_efficient_cpu: 35274

Delayed dequeue on (3 iterations):
------------------------------------------------

Calls to find_energy_efficient_cpu: 204079 Overutilized when in find_energy_efficient_cpu: 86966

Calls to find_energy_efficient_cpu: 207351 Overutilized when in find_energy_efficient_cpu: 111233

Calls to find_energy_efficient_cpu: 208556 Overutilized when in find_energy_efficient_cpu: 77986

Delayed dequeue off again (3 iterations):
------------------------------------------------

Calls to find_energy_efficient_cpu: 210396 Overutilized when in find_energy_efficient_cpu: 34108

Calls to find_energy_efficient_cpu: 210978 Overutilized when in find_energy_efficient_cpu: 33122

Calls to find_energy_efficient_cpu: 210181 Overutilized when in find_energy_efficient_cpu: 33060

Delayed dequeue on again (3 iterations):
------------------------------------------------

Calls to find_energy_efficient_cpu: 202932 Overutilized when in find_energy_efficient_cpu: 35060

Calls to find_energy_efficient_cpu: 206098 Overutilized when in find_energy_efficient_cpu: 57373

Calls to find_energy_efficient_cpu: 199444 Overutilized when in find_energy_efficient_cpu: 33959

On the matter of switched_from_fair() being called with a task that is sched_delayed == 1, I can confirm I see some occurrences of that. They are not that numerous, but do happen from time to time on the trace data. I haven't investigated things in that direction yet though.

But we do make those tasks sched_delayed = 0 in there, so things should go back to normal eventually, unless we fail to dequeue the sched_delayed task properly for this case.

Quoted 14 lines

So I re-read all this util_est stuff again this evening and I am confused :-) Please bear with me.

So the old code does:

	dequeue_task_fair()
	  util_est_dequeue();
	  // actual dequeue -- updates pelt
	  util_est_update();

While the new code does:

Quoted 10 lines
	dequeue_task_fair()
	  util_est_dequeue();
	  if (!dequeue())
	    return;
	  util_est_update();

	delayed_dequeue:
	  util_est_update();

The above is my understanding as well.

Quoted 6 lines

Specifically, we only call util_est_update() if/when we do the actual dequeue -- potentially at a later point in time. Because, as I argued in the comments, ttwu()'s ENQUEUE_DELAYED will not actually enqueue the task (since it is still enqueued) and therefore the update would be spurious.

Quoted 7 lines

However!!, if we do dequeue, then we'll end up updating the EWMA with a potentially different task_util(p).

And this is where the confusion comes... this extra time on the runqueue would not be running and thus decreate util_avg, as such task_util_est() should be lower than before and tasks should tend towards smaller cores, rather than larger cores as you seem to be seeing.

Right. I also expected the util_avg to decrease with more time on the runqueue. And that might be happening...

Quoted 29 lines

[ there is the 'dequeued + UTIL_EST_MARGIN < task_runnable()' escape
clause, which we might be hitting... dunno ]

In any case, if you haven't tried it already, could I ask you to test the below (once you're back in the office)?

Also, this doesn't really explain why things go sideways once you enable DELAY_DEQUEUE and then don't re-align if you disable it again. I mean, it should eventually age out the dodgy EWMA contributions and start working as expected.

kernel/sched/fair.c +2 -2
@@ -7008,7 +7008,6 @@ static int dequeue_entities(struct rq *r
 			SCHED_WARN_ON(p->on_rq != 1);
 
 			/* Fix-up what dequeue_task_fair() skipped */
-			util_est_update(&rq->cfs, p, task_sleep);
 			hrtick_update(rq);
 
 			/* Fix-up what block_task() skipped. */
@@ -7028,13 +7027,11 @@ static bool dequeue_task_fair(struct rq
 {
 	util_est_dequeue(&rq->cfs, p);
 
-	if (dequeue_entities(rq, &p->se, flags) < 0)
+	if (dequeue_entities(rq, &p->se, flags) < 0) {
+		util_est_update(&rq->cfs, p, DEQUEUE_SLEEP);

kernel/sched/fair.c static bool dequeue_task_fair(struct rq · line 7037

Quoted 10 lines
kernel/sched/fair.c +1 -4
 		return false;
+	}
 
-	/*
-	 * It doesn't make sense to update util_est for the delayed dequeue
-	 * case where ttwu will make it appear the sleep never happened.
-	 */
 	util_est_update(&rq->cfs, p, flags & DEQUEUE_SLEEP);
 	hrtick_update(rq);
 	return true;

... as the above patch unfortunately doesn't improve things. The util_est_update might be a red herring here.

Exchanging some information with Hongyan today, he was a bit suspicious of the uclamp behavior with the eevdf complete series applied.

Checking the uclamp code, I see we have some refcounting tied to enqueuing/dequeuing of tasks, and the uclamp values are organized in buckets.

Just for fun I added a few trace_printk's in uclamp_eff_value, uclamp_rq_inc_id and uclamp_rq_dec_id.

Booting up the system with delayed_dequeue disabled and running the benchmark, I see the uclamp bucket management pretty stable. Tasks get added to the uclamp buckets but then get removed. At the end of the benchmark, the uclamp buckets are (almost always) clean of tasks.

Enabling delayed dequeue, I can see the uclamp buckets slowly filling up with tasks. At the end of the benchmark, I see uclamp buckets with 30, 40 or 50 tasks still. If I do another run, I can see 80, 100 tasks still.

I suspect refcounting might be going wrong somewhere due to delayed dequeue tasks, but that's more of a guess right now. Hopefully that is useful information. I'll resume investigation tomorrow.

author maintainer
#45 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 14 lines · Luis Machado

Exchanging some information with Hongyan today, he was a bit suspicious of the uclamp behavior with the eevdf complete series applied.

Checking the uclamp code, I see we have some refcounting tied to enqueuing/dequeuing of tasks, and the uclamp values are organized in buckets.

Just for fun I added a few trace_printk's in uclamp_eff_value, uclamp_rq_inc_id and uclamp_rq_dec_id.

Booting up the system with delayed_dequeue disabled and running the benchmark, I see the uclamp bucket management pretty stable. Tasks get added to the uclamp buckets but then get removed. At the end of the benchmark, the uclamp buckets are (almost always) clean of tasks.

Quoted 7 lines

Enabling delayed dequeue, I can see the uclamp buckets slowly filling up with tasks. At the end of the benchmark, I see uclamp buckets with 30, 40 or 50 tasks still. If I do another run, I can see 80, 100 tasks still.

I suspect refcounting might be going wrong somewhere due to delayed dequeue tasks, but that's more of a guess right now. Hopefully that is useful information. I'll resume investigation tomorrow.

Thank you both!!

Does the below help?

Note how dequeue_task() does uclamp_rq_dec() unconditionally, which is then not balanced in the case below.

kernel/sched/core.c +1 -0
@@ -3664,6 +3664,7 @@ static int ttwu_runnable(struct task_str
 			/* mustn't run a delayed task */
 			SCHED_WARN_ON(task_on_cpu(rq, p));
 			enqueue_task(rq, p, ENQUEUE_DELAYED);
+			uclamp_rq_inc(rq, p);
 		}
 		if (!task_on_cpu(rq, p)) {
 			/*
#46 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Hi Peter,

Quoted 34 lines · Peter Zijlstra
Quoted 14 lines · Luis Machado

Exchanging some information with Hongyan today, he was a bit suspicious of the uclamp behavior with the eevdf complete series applied.

Checking the uclamp code, I see we have some refcounting tied to enqueuing/dequeuing of tasks, and the uclamp values are organized in buckets.

Just for fun I added a few trace_printk's in uclamp_eff_value, uclamp_rq_inc_id and uclamp_rq_dec_id.

Booting up the system with delayed_dequeue disabled and running the benchmark, I see the uclamp bucket management pretty stable. Tasks get added to the uclamp buckets but then get removed. At the end of the benchmark, the uclamp buckets are (almost always) clean of tasks.

Quoted 7 lines

Enabling delayed dequeue, I can see the uclamp buckets slowly filling up with tasks. At the end of the benchmark, I see uclamp buckets with 30, 40 or 50 tasks still. If I do another run, I can see 80, 100 tasks still.

I suspect refcounting might be going wrong somewhere due to delayed dequeue tasks, but that's more of a guess right now. Hopefully that is useful information. I'll resume investigation tomorrow.

Thank you both!!

Does the below help?

Note how dequeue_task() does uclamp_rq_dec() unconditionally, which is then not balanced in the case below.

kernel/sched/core.c static int ttwu_runnable(struct task_str · line 3670

Quoted 7 lines
kernel/sched/core.c +1 -0
@@ -3664,6 +3664,7 @@ static int ttwu_runnable(struct task_str
  			/* mustn't run a delayed task */
  			SCHED_WARN_ON(task_on_cpu(rq, p));
  			enqueue_task(rq, p, ENQUEUE_DELAYED);
+			uclamp_rq_inc(rq, p);
  		}
  		if (!task_on_cpu(rq, p)) {
  			/*

Further communication with Luis makes me investigate more on uclamp bucket leaks.

I added a separate variable uclamp_on_rq in task_struct and WARN when the state didn't match. The thing is, without your above fix, the first WARN happens on uclamp_rq_inc(), not dec(), which means a task is already added to rq uclamp and is somehow added again by mistake.

Because this is a double inc(), your fix won't fix this problem. I added the trace here when the double inc() happened, in case you can see something.

Juno-r2-Debian login: [ 176.010780] ------------[ cut here ]------------

                            [1083/15764]
[  176.015414] WARNING: CPU: 5 PID: 0 at kernel/sched/core.c:1594 uclamp_rq_inc+0x188/0x1a0 
[  176.023539] Modules linked in: 
[  176.026602] CPU: 5 PID: 0 Comm: swapper/5 Not tainted 6.10.0-rc1-ga946a2b48119-dirty #1034 
[  176.034882] Hardware name: ARM Juno development board (r2) (DT) 
[  176.040815] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) 
[  176.047796] pc : uclamp_rq_inc+0x188/0x1a0 
[  176.051903] lr : enqueue_task+0x48/0x70 
[  176.055752] sp : ffffffc081fb3de0 
[  176.059073] x29: ffffffc081fb3de0 x28: 0000000000000000 x27: ffffff8800342300 
[  176.066233] x26: ffffff88084311b8 x25: ffffffc081ed2000 x24: ffffffc081ca7008 
[  176.073388] x23: 0000000000000000 x22: 00000000000000c0 x21: ffffff897efbbf40 
[  176.080543] x20: ffffff8808431180 x19: 0000000000000049 x18: 0000000000000408 
[  176.087700] x17: ffffffc8fd301000 x16: ffffffc081fb0000 x15: 000000000000afc8 
[  176.094857] x14: 0000000000000001 x13: 0000000000000004 x12: 000000020c8392c2 
[  176.102017] x11: 0000000000000800 x10: 0000000000000800 x9 : 0000000000000000 
[  176.109177] x8 : 0000000000000001 x7 : 0000000000000021 x6 : ffffffffffffff4c 
[  176.116337] x5 : 0000000000007e90 x4 : 00000000000fc33c x3 : 00000000000139b4 
[  176.123487] x2 : 0000000000000049 x1 : ffffff8808431180 x0 : ffffff897efbbf40 
[  176.130639] Call trace: 
[  176.133089]  uclamp_rq_inc+0x188/0x1a0 
[  176.136849]  activate_task+0x48/0x60 
[  176.140433]  ttwu_do_activate+0x80/0x22c 
[  176.144364]  sched_ttwu_pending+0x9c/0x120 
[  176.148470]  __flush_smp_call_function_queue+0x3d0/0x568 
[  176.153792]  generic_smp_call_function_single_interrupt+0x10/0x18 
[  176.159902]  do_handle_IPI+0xa0/0x230 
[  176.163577]  ipi_handler+0x18/0x24 
[  176.166987]  handle_percpu_devid_irq+0x80/0x204 
[  176.171532]  generic_handle_domain_irq+0x28/0x40 
[  176.176161]  gic_handle_irq+0x58/0xd0 
[  176.179833]  call_on_irq_stack+0x24/0x34 
[  176.183764]  el1_interrupt+0x70/0xbc 
[  176.187348]  el1h_64_irq_handler+0x14/0x1c 
[  176.191449]  el1h_64_irq+0x64/0x68 
[  176.194857]  arch_local_irq_enable+0x4/0x8 
[  176.198960]  cpuidle_enter+0x34/0x48 
[  176.202547]  do_idle+0x1c4/0x234 
[  176.205786]  cpu_startup_entry+0x30/0x34 
[  176.209723]  secondary_start_kernel+0x124/0x144 
[  176.214267]  __secondary_switched+0x6c/0x70 
[  176.218458] ---[ end trace 0000000000000000 ]---

Or, do you mean to add a uclamp_rq_dec() instead? But even if I skip the uclamp_rq_inc() at that line (which is equivalent to doing uclamp_rq_dec() after enqueue_task()), I still get a WARN like this:

Juno-r2-Debian login: [ 77.183511] ------------[ cut here ]------------

[   77.188157] WARNING: CPU: 2 PID: 114 at kernel/sched/core.c:1594 uclamp_rq_inc+0x188/0x1a0 
[   77.196455] Modules linked in: 
[   77.199519] CPU: 2 PID: 114 Comm: kworker/u25:3 Not tainted 6.10.0-rc1-ga946a2b48119-dirty #1035 
[   77.208325] Hardware name: ARM Juno development board (r2) (DT) 
[   77.214255] Workqueue: rpciod rpc_async_schedule 
[   77.218893] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) 
[   77.225873] pc : uclamp_rq_inc+0x188/0x1a0 
[   77.229979] lr : enqueue_task+0x44/0x7c 
[   77.233826] sp : ffffffc082af3c80 
[   77.237143] x29: ffffffc082af3c80 x28: ffffffc081cd6000 x27: 0000000000000402 
[   77.244305] x26: ffffffc8fd285000 x25: ffffffc081cbaf40 x24: ffffff8800f92300 
[   77.251466] x23: ffffff897ef3ff40 x22: 00000000000000c0 x21: ffffff897ef3ff40 
[   77.258625] x20: ffffff88028f3480 x19: 0000000000000009 x18: 0000000000000000 
[   77.265784] x17: 0000000000000000 x16: 0000000000000001 x15: 00000000000003ae 
[   77.272943] x14: 000000000003eca5 x13: 000000000003eca5 x12: 00000000000000eb 
[   77.280102] x11: 0000000000000000 x10: 0000000000001800 x9 : 0000000000000000 
[   77.287261] x8 : 0000000000000001 x7 : 0000000000000000 x6 : 000000000000003f 
[   77.294420] x5 : 0000000000000311 x4 : 0000000000000113 x3 : 000000000000017f 
[   77.301578] x2 : 0000000000000009 x1 : ffffff88028f3480 x0 : ffffff897ef3ff40 
[   77.308738] Call trace: 
[   77.311185]  uclamp_rq_inc+0x188/0x1a0 
[   77.314944]  activate_task+0x48/0x60 
[   77.318530]  ttwu_do_activate+0x80/0x22c 
[   77.322464]  try_to_wake_up+0x1e8/0x50c 
[   77.326311]  wake_up_process+0x14/0x1c 
[   77.330070]  kick_pool+0xb8/0x144 
[   77.333395]  process_one_work+0x104/0x3a4 
[   77.337418]  worker_thread+0x2b8/0x424 
[   77.341176]  kthread+0xf8/0x194 
[   77.344327]  ret_from_fork+0x10/0x20 
[   77.347913] ---[ end trace 0000000000000000 ]---

I pasted the trace here in case useful.

Hongyan

#47 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra
Quoted 34 lines · Peter Zijlstra
Quoted 14 lines · Luis Machado

Exchanging some information with Hongyan today, he was a bit suspicious of the uclamp behavior with the eevdf complete series applied.

Checking the uclamp code, I see we have some refcounting tied to enqueuing/dequeuing of tasks, and the uclamp values are organized in buckets.

Just for fun I added a few trace_printk's in uclamp_eff_value, uclamp_rq_inc_id and uclamp_rq_dec_id.

Booting up the system with delayed_dequeue disabled and running the benchmark, I see the uclamp bucket management pretty stable. Tasks get added to the uclamp buckets but then get removed. At the end of the benchmark, the uclamp buckets are (almost always) clean of tasks.

Quoted 7 lines

Enabling delayed dequeue, I can see the uclamp buckets slowly filling up with tasks. At the end of the benchmark, I see uclamp buckets with 30, 40 or 50 tasks still. If I do another run, I can see 80, 100 tasks still.

I suspect refcounting might be going wrong somewhere due to delayed dequeue tasks, but that's more of a guess right now. Hopefully that is useful information. I'll resume investigation tomorrow.

Thank you both!!

Does the below help?

Note how dequeue_task() does uclamp_rq_dec() unconditionally, which is then not balanced in the case below.

kernel/sched/core.c static int ttwu_runnable(struct task_str · line 3670

Quoted 7 lines
kernel/sched/core.c +1 -0
@@ -3664,6 +3664,7 @@ static int ttwu_runnable(struct task_str
 			/* mustn't run a delayed task */
 			SCHED_WARN_ON(task_on_cpu(rq, p));
 			enqueue_task(rq, p, ENQUEUE_DELAYED);
+			uclamp_rq_inc(rq, p);
 		}
 		if (!task_on_cpu(rq, p)) {
 			/*

As Hongyan pointed out in a separate message, the above makes things worse, as we end up with even more leftover tasks in the uclamp buckets.

I'm trying a fix in kernel/sched/core.c:enqueue_task that only calls uclamp_rq_inc if the task is not sched_delayed, so:

- uclamp_rq_inc(rq, p);
+ if (!p->se.sched_delayed)
+ uclamp_rq_inc(rq, p);

I'm not entirely sure it is correct, but it seems to fix things, but I'm still running some tests.

With the current code, given uclamp_rq_inc and uclamp_rq_dec get called in enqueue_task and dequeue_task, the additional enqueue_task call from ttwu_runnable for a delayed_dequeue task may do an additional unconditional call to uclamp_rq_inc, no?

#48 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 54 lines · Luis Machado
Quoted 34 lines · Peter Zijlstra
Quoted 14 lines · Luis Machado

Exchanging some information with Hongyan today, he was a bit suspicious of the uclamp behavior with the eevdf complete series applied.

Checking the uclamp code, I see we have some refcounting tied to enqueuing/dequeuing of tasks, and the uclamp values are organized in buckets.

Just for fun I added a few trace_printk's in uclamp_eff_value, uclamp_rq_inc_id and uclamp_rq_dec_id.

Booting up the system with delayed_dequeue disabled and running the benchmark, I see the uclamp bucket management pretty stable. Tasks get added to the uclamp buckets but then get removed. At the end of the benchmark, the uclamp buckets are (almost always) clean of tasks.

Quoted 7 lines

Enabling delayed dequeue, I can see the uclamp buckets slowly filling up with tasks. At the end of the benchmark, I see uclamp buckets with 30, 40 or 50 tasks still. If I do another run, I can see 80, 100 tasks still.

I suspect refcounting might be going wrong somewhere due to delayed dequeue tasks, but that's more of a guess right now. Hopefully that is useful information. I'll resume investigation tomorrow.

Thank you both!!

Does the below help?

Note how dequeue_task() does uclamp_rq_dec() unconditionally, which is then not balanced in the case below.

kernel/sched/core.c static int ttwu_runnable(struct task_str · line 3670

Quoted 7 lines
kernel/sched/core.c +1 -0
@@ -3664,6 +3664,7 @@ static int ttwu_runnable(struct task_str
  			/* mustn't run a delayed task */
  			SCHED_WARN_ON(task_on_cpu(rq, p));
  			enqueue_task(rq, p, ENQUEUE_DELAYED);
+			uclamp_rq_inc(rq, p);
  		}
  		if (!task_on_cpu(rq, p)) {
  			/*

As Hongyan pointed out in a separate message, the above makes things worse, as we end up with even more leftover tasks in the uclamp buckets.

I'm trying a fix in kernel/sched/core.c:enqueue_task that only calls uclamp_rq_inc if the task is not sched_delayed, so:

(quoted diff) +2 -1
-       uclamp_rq_inc(rq, p);
+       if (!p->se.sched_delayed)
+         uclamp_rq_inc(rq, p);
Quoted 2 lines

I'm not entirely sure it is correct, but it seems to fix things, but I'm still running some tests.

This seems to so far not trigger any WARN or bad behavior in my tests.

Quoted 5 lines

With the current code, given uclamp_rq_inc and uclamp_rq_dec get called in enqueue_task and dequeue_task, the additional enqueue_task call from ttwu_runnable for a delayed_dequeue task may do an additional unconditional call to uclamp_rq_inc, no?

author maintainer
#49 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Luis Machado
Quoted 27 lines · Luis Machado

kernel/sched/core.c static int ttwu_runnable(struct task_str · line 3670

Quoted 13 lines · Peter Zijlstra

Note how dequeue_task() does uclamp_rq_dec() unconditionally, which is then not balanced in the case below.

kernel/sched/core.c +1 -0
@@ -3664,6 +3664,7 @@ static int ttwu_runnable(struct task_str
 			/* mustn't run a delayed task */
 			SCHED_WARN_ON(task_on_cpu(rq, p));
 			enqueue_task(rq, p, ENQUEUE_DELAYED);
+			uclamp_rq_inc(rq, p);
 		}
 		if (!task_on_cpu(rq, p)) {
 			/*

As Hongyan pointed out in a separate message, the above makes things worse, as we end up with even more leftover tasks in the uclamp buckets.

I'm trying a fix in kernel/sched/core.c:enqueue_task that only calls uclamp_rq_inc if the task is not sched_delayed, so:

(quoted diff) +2 -1
-       uclamp_rq_inc(rq, p);
+       if (!p->se.sched_delayed)
+         uclamp_rq_inc(rq, p);
Quoted 7 lines

I'm not entirely sure it is correct, but it seems to fix things, but I'm still running some tests.

With the current code, given uclamp_rq_inc and uclamp_rq_dec get called in enqueue_task and dequeue_task, the additional enqueue_task call from ttwu_runnable for a delayed_dequeue task may do an additional unconditional call to uclamp_rq_inc, no?

Yes, I got enqueue_task() and class->enqueue_task() confused this morning.

But with the above, you skip inc for sched_delayed, but dequeue_task() will have done the dec, so isn't it then still unbalanced?

Oh well, I'll go stare at this in tomorrow.

In any case, is there a uclamp self-test somewhere?

author maintainer
#50 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 4 lines · Peter Zijlstra

But with the above, you skip inc for sched_delayed, but dequeue_task() will have done the dec, so isn't it then still unbalanced?

Oh well, I'll go stare at this in tomorrow.

OK, just before I went to play in my giant hamster wheel it hit me.

When dequeue_task() 'fails' and sets sched_delayed, we'll have done uclamp_rq_dec().

Then, since the delayed task is still on the rq -- per the failure -- it can be migrated, this will again do dequeue_task() which will *agian* do a uclamp_rq_dec().

So now we have a double dequeue -- *FAIL*.

Worse, the migration will then do an enqueue_task() on the new rq causing uclamp_rq_inc(). If you then get a ttwu() / ENQUEUE_DELAYED, you can tickle yet another uclamp_rq_inc() for another fail.

Something like the below avoids uclamp_rq_{inc,dec}() when ->sched_delayed, and moves it post class->enqueue_task() such that for the ENQUEUE_DELAYED case, we *will* do the inc after ->sched_delayed gets cleared.

Hmm?

---

kernel/sched/core.c +9 -1
index 9ac1054c2a4bb..965e6464e68e9 100644
@@ -1676,6 +1676,9 @@ static inline void uclamp_rq_inc(struct rq *rq, struct task_struct *p)
 	if (unlikely(!p->sched_class->uclamp_enabled))
 		return;
 
+	if (p->se.sched_delayed)
+		return;
+
 	for_each_clamp_id(clamp_id)
 		uclamp_rq_inc_id(rq, p, clamp_id);
 
@@ -1700,6 +1703,9 @@ static inline void uclamp_rq_dec(struct rq *rq, struct task_struct *p)
 	if (unlikely(!p->sched_class->uclamp_enabled))
 		return;
 
+	if (p->se.sched_delayed)
+		return;
+
 	for_each_clamp_id(clamp_id)
 		uclamp_rq_dec_id(rq, p, clamp_id);
 }
@@ -1979,9 +1985,10 @@ void enqueue_task(struct rq *rq, struct task_struct *p, int flags)
 		psi_enqueue(p, (flags & ENQUEUE_WAKEUP) && !(flags & ENQUEUE_MIGRATED));
 	}
 
-	uclamp_rq_inc(rq, p);
 	p->sched_class->enqueue_task(rq, p, flags);
 
+	uclamp_rq_inc(rq, p);
+
 	if (sched_core_enabled(rq))
 		sched_core_enqueue(rq, p);
 }
@@ -2003,6 +2010,7 @@ bool dequeue_task(struct rq *rq, struct task_struct *p, int flags)
 	}
 
 	uclamp_rq_dec(rq, p);
+
 	return p->sched_class->dequeue_task(rq, p, flags);
 }
 
#51 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Peter Zijlstra

But with the above, you skip inc for sched_delayed, but dequeue_task() will have done the dec, so isn't it then still unbalanced?

Oh well, I'll go stare at this in tomorrow.

It did not appear to keep things unbalanced towards uclamp_rq_dec, but might've been luck of not hitting the right code path.

In any case ...

Quoted 61 lines

OK, just before I went to play in my giant hamster wheel it hit me.

When dequeue_task() 'fails' and sets sched_delayed, we'll have done uclamp_rq_dec().

Then, since the delayed task is still on the rq -- per the failure -- it can be migrated, this will again do dequeue_task() which will *agian* do a uclamp_rq_dec().

So now we have a double dequeue -- *FAIL*.

Worse, the migration will then do an enqueue_task() on the new rq causing uclamp_rq_inc(). If you then get a ttwu() / ENQUEUE_DELAYED, you can tickle yet another uclamp_rq_inc() for another fail.

Something like the below avoids uclamp_rq_{inc,dec}() when ->sched_delayed, and moves it post class->enqueue_task() such that for the ENQUEUE_DELAYED case, we *will* do the inc after ->sched_delayed gets cleared.

Hmm?

---

kernel/sched/core.c +8 -1
index 9ac1054c2a4bb..965e6464e68e9 100644
@@ -1676,6 +1676,9 @@ static inline void uclamp_rq_inc(struct rq *rq, struct task_struct *p)
 	if (unlikely(!p->sched_class->uclamp_enabled))
 		return;
 
+	if (p->se.sched_delayed)
+		return;
+
 	for_each_clamp_id(clamp_id)
 		uclamp_rq_inc_id(rq, p, clamp_id);
 
@@ -1700,6 +1703,9 @@ static inline void uclamp_rq_dec(struct rq *rq, struct task_struct *p)
 	if (unlikely(!p->sched_class->uclamp_enabled))
 		return;
 
+	if (p->se.sched_delayed)
+		return;
+
 	for_each_clamp_id(clamp_id)
 		uclamp_rq_dec_id(rq, p, clamp_id);
 }
@@ -1979,9 +1985,10 @@ void enqueue_task(struct rq *rq, struct task_struct *p, int flags)
 		psi_enqueue(p, (flags & ENQUEUE_WAKEUP) && !(flags & ENQUEUE_MIGRATED));
 	}
 
-	uclamp_rq_inc(rq, p);
 	p->sched_class->enqueue_task(rq, p, flags);
 
+	uclamp_rq_inc(rq, p);
+
 	if (sched_core_enabled(rq))
 		sched_core_enqueue(rq, p);
 }

kernel/sched/core.c bool dequeue_task(struct rq *rq, struct task_struct *p, int flags) · line 2016

Quoted 7 lines
kernel/sched/core.c +1 -0
@@ -2003,6 +2010,7 @@ bool dequeue_task(struct rq *rq, struct task_struct *p, int flags)
 	}
 
 	uclamp_rq_dec(rq, p);
+
 	return p->sched_class->dequeue_task(rq, p, flags);
 }
 

... thanks for the patch! The above seems to do it for me. I can see more reasonable energy use with the eevdf-complete series. Still a bit higher. Might be noise, we'll see.

I'll go stare at it and run some more tests on our end with this fix applied on top and will report back.

author maintainer
#52 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 3 lines · Luis Machado

... thanks for the patch! The above seems to do it for me. I can see more reasonable energy use with the eevdf-complete series. Still a bit higher. Might be noise, we'll see.

W00t!!!

Let me write a decent Changelog and stuff it in the git tree along with all the other bits.

Thanks for all the testing.

#53 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 11 lines · Peter Zijlstra
Quoted 3 lines · Luis Machado

... thanks for the patch! The above seems to do it for me. I can see more reasonable energy use with the eevdf-complete series. Still a bit higher. Might be noise, we'll see.

W00t!!!

Let me write a decent Changelog and stuff it in the git tree along with all the other bits.

Thanks for all the testing.

I've been doing some more testing of the eevdf-complete series with the Pixel6/EAS platform. Hopefully these numbers will prove useful.

The energy regression from the original delayed-dequeue patch seems to have been mostly fixed (with your proposed patch). Energy readings for the big and mid cores are mostly stable and comparable with stock eevdf (without eevdf-complete).

The only difference I can spot now is in the energy use of the little cores. Compared to stock eevdf, the delayed-dequeue code seems to make the energy use of the little cores a bit spiky, meaning sometimes we get the expected level of energy use, but other times we get 40%, 60% or even 90% more.

For instance...

(1) m6.6-eevdf-stock*: stock eevdf runs
(2) m6.6-eevdf-complete-ndd-dz*: eevdf-complete + NO_DELAY_DEQUEUE + DELAY_ZERO
(3) m6.6-eevdf-complete-dd-dz*: eevdf-complete + DELAY_DEQUEUE + DELAY_ZERO

+------------+---------------------------------+-----------+
|  channel   |                      tag        | perc_diff |
+------------+---------------------------------+-----------+
| CPU-Little |  m6.6-eevdf-stock-1             |   0.0%    |
| CPU-Little |  m6.6-eevdf-stock-2             |  -4.21%   |
| CPU-Little |  m6.6-eevdf-stock-3             |  -7.86%   |
| CPU-Little |  m6.6-eevdf-stock-4             |  -5.67%   |
| CPU-Little |  m6.6-eevdf-stock-5             |  -6.61%   |
| CPU-Little |  m6.6-eevdf-complete-ndd-dz-1   |  -2.21%   |
| CPU-Little |  m6.6-eevdf-complete-ndd-dz-2   |  -9.99%   |
| CPU-Little |  m6.6-eevdf-complete-ndd-dz-3   |   -6.1%   |
| CPU-Little |  m6.6-eevdf-complete-ndd-dz-4   |  -5.66%   |
| CPU-Little |  m6.6-eevdf-complete-ndd-dz-5   |  -7.12%   |
| CPU-Little |  m6.6-eevdf-complete-dd-dz-1    |  96.69%   |
| CPU-Little |  m6.6-eevdf-complete-dd-dz-2    |   22.1%   |
| CPU-Little |  m6.6-eevdf-complete-dd-dz-3    |  44.82%   |
| CPU-Little |  m6.6-eevdf-complete-dd-dz-4    |  -0.23%   |
| CPU-Little |  m6.6-eevdf-complete-dd-dz-5    |   8.28%   |
+------------+---------------------------------+-----------+

Looking at what might explain the spiky behavior with DELAY_DEQUEUE, I noticed the idle residency data (we have 2 idle states) also shows some spikyness and potential clues.

Looks like (1) and (2) manage to switch to idle states in a consistent manner, whereas (3) seems a bit erratic and more prone to take a shallower idle state (idle 0) as opposed to a deeper idle state (idle 1).

(1) and (2) seem to make better use of the deeper idle state.

+-------------------------------+---------+------------+-------+
|            tag                | cluster | idle_state | time  |
+-------------------------------+---------+------------+-------+
| m6.6-eevdf-stock-1            | little  |  not idle  | 63.49 |
| m6.6-eevdf-stock-1            | little  |   idle 0   | 30.66 |
| m6.6-eevdf-stock-1            | little  |   idle 1   | 12.15 |
| m6.6-eevdf-stock-2            | little  |  not idle  | 62.6  |
| m6.6-eevdf-stock-2            | little  |   idle 0   | 31.13 |
| m6.6-eevdf-stock-2            | little  |   idle 1   | 14.56 |
| m6.6-eevdf-stock-3            | little  |  not idle  | 63.98 |
| m6.6-eevdf-stock-3            | little  |   idle 0   | 31.54 |
| m6.6-eevdf-stock-3            | little  |   idle 1   | 15.91 |
| m6.6-eevdf-stock-4            | little  |  not idle  | 64.18 |
| m6.6-eevdf-stock-4            | little  |   idle 0   | 31.32 |
| m6.6-eevdf-stock-4            | little  |   idle 1   | 15.83 |
| m6.6-eevdf-stock-5            | little  |  not idle  | 63.32 |
| m6.6-eevdf-stock-5            | little  |   idle 0   | 30.4  |
| m6.6-eevdf-stock-5            | little  |   idle 1   | 14.33 |
| m6.6-eevdf-complete-ndd-dz-1  | little  |  not idle  | 62.62 |
| m6.6-eevdf-complete-ndd-dz-1  | little  |   idle 0   | 29.48 |
| m6.6-eevdf-complete-ndd-dz-1  | little  |   idle 1   | 13.19 |
| m6.6-eevdf-complete-ndd-dz-2  | little  |  not idle  | 64.12 |
| m6.6-eevdf-complete-ndd-dz-2  | little  |   idle 0   | 27.62 |
| m6.6-eevdf-complete-ndd-dz-2  | little  |   idle 1   | 14.73 |
| m6.6-eevdf-complete-ndd-dz-3  | little  |  not idle  | 62.86 |
| m6.6-eevdf-complete-ndd-dz-3  | little  |   idle 0   | 27.87 |
| m6.6-eevdf-complete-ndd-dz-3  | little  |   idle 1   | 14.97 |
| m6.6-eevdf-complete-ndd-dz-4  | little  |  not idle  | 63.01 |
| m6.6-eevdf-complete-ndd-dz-4  | little  |   idle 0   | 28.2  |
| m6.6-eevdf-complete-ndd-dz-4  | little  |   idle 1   | 14.11 |
| m6.6-eevdf-complete-ndd-dz-5  | little  |  not idle  | 62.1  |
| m6.6-eevdf-complete-ndd-dz-5  | little  |   idle 0   | 29.06 |
| m6.6-eevdf-complete-ndd-dz-5  | little  |   idle 1   | 14.73 |
| m6.6-eevdf-complete-dd-dz-1   | little  |  not idle  | 46.18 |
| m6.6-eevdf-complete-dd-dz-1   | little  |   idle 0   | 53.78 |
| m6.6-eevdf-complete-dd-dz-1   | little  |   idle 1   | 3.75  |
| m6.6-eevdf-complete-dd-dz-2   | little  |  not idle  | 57.64 |
| m6.6-eevdf-complete-dd-dz-2   | little  |   idle 0   | 40.47 |
| m6.6-eevdf-complete-dd-dz-2   | little  |   idle 1   | 7.39  |
| m6.6-eevdf-complete-dd-dz-3   | little  |  not idle  | 43.14 |
| m6.6-eevdf-complete-dd-dz-3   | little  |   idle 0   | 57.73 |
| m6.6-eevdf-complete-dd-dz-3   | little  |   idle 1   | 3.65  |
| m6.6-eevdf-complete-dd-dz-4   | little  |  not idle  | 58.97 |
| m6.6-eevdf-complete-dd-dz-4   | little  |   idle 0   | 36.4  |
| m6.6-eevdf-complete-dd-dz-4   | little  |   idle 1   | 9.42  |
| m6.6-eevdf-complete-dd-dz-5   | little  |  not idle  | 55.85 |
| m6.6-eevdf-complete-dd-dz-5   | little  |   idle 0   | 36.75 |
| m6.6-eevdf-complete-dd-dz-5   | little  |   idle 1   | 13.14 |
+-------------------------------+---------+------------+-------+

I can't draw a precise conclusion, but it might be down to delayed util_est updates or even the additional time the delayed-dequeue tasks spend on the runqueue. But delayed-dequeue does change the overall behavior a bit on these heterogeneous platforms, energy-wise.

author maintainer
#54 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Luis Machado
Quoted 13 lines · Luis Machado

(2) m6.6-eevdf-complete: m6.6-stock plus this series.
(3) m6.6-eevdf-complete-no-delay-dequeue: (2) + NO_DELAY_DEQUEUE

+------------+------------------------------------------------------+-----------+
|  cluster   |                         tag                          | perc_diff |
+------------+------------------------------------------------------+-----------+
|    CPU     |                   m6.6-stock                         |   0.0%    |
|  CPU-Big   |                   m6.6-stock                         |   0.0%    |
| CPU-Little |                   m6.6-stock                         |   0.0%    |
|  CPU-Mid   |                   m6.6-stock                         |   0.0%    |
|    GPU     |                   m6.6-stock                         |   0.0%    |
|   Total    |                   m6.6-stock                         |   0.0%    |
Quoted 6 lines
|    CPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  117.77%  |
|  CPU-Big   |        m6.6-eevdf-complete-no-delay-dequeue          |  113.79%  |
| CPU-Little |        m6.6-eevdf-complete-no-delay-dequeue          |  97.47%   |
|  CPU-Mid   |        m6.6-eevdf-complete-no-delay-dequeue          |  189.0%   |
|    GPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  -6.74%   |
|   Total    |        m6.6-eevdf-complete-no-delay-dequeue          |  103.84%  |

This one is still flummoxing me. I've gone over the patch a few times on different days and I'm not seeing it. Without DELAY_DEQUEUE it should behave as before.

Let me try and split this patch up into smaller parts such that you can try and bisect this.

#55 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Peter,

Quoted 22 lines · Peter Zijlstra
Quoted 13 lines · Luis Machado

(2) m6.6-eevdf-complete: m6.6-stock plus this series.
(3) m6.6-eevdf-complete-no-delay-dequeue: (2) + NO_DELAY_DEQUEUE

+------------+------------------------------------------------------+-----------+
|  cluster   |                         tag                          | perc_diff |
+------------+------------------------------------------------------+-----------+
|    CPU     |                   m6.6-stock                         |   0.0%    |
|  CPU-Big   |                   m6.6-stock                         |   0.0%    |
| CPU-Little |                   m6.6-stock                         |   0.0%    |
|  CPU-Mid   |                   m6.6-stock                         |   0.0%    |
|    GPU     |                   m6.6-stock                         |   0.0%    |
|   Total    |                   m6.6-stock                         |   0.0%    |
Quoted 6 lines
|    CPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  117.77%  |
|  CPU-Big   |        m6.6-eevdf-complete-no-delay-dequeue          |  113.79%  |
| CPU-Little |        m6.6-eevdf-complete-no-delay-dequeue          |  97.47%   |
|  CPU-Mid   |        m6.6-eevdf-complete-no-delay-dequeue          |  189.0%   |
|    GPU     |        m6.6-eevdf-complete-no-delay-dequeue          |  -6.74%   |
|   Total    |        m6.6-eevdf-complete-no-delay-dequeue          |  103.84%  |
Quoted 7 lines

This one is still flummoxing me. I've gone over the patch a few times on different days and I'm not seeing it. Without DELAY_DEQUEUE it should behave as before.

Let me try and split this patch up into smaller parts such that you can try and bisect this.

Same situation on my end. I've been chasing this for some time and I don't fully understand why things go off the rails energy-wise as soon as DELAY_DEQUEUE is enabled, now that the load_avg accounting red herring is gone.

I do have one additional piece of information though. Hopefully it will be useful.

Booting the kernel with NO_DELAY_DEQUEUE (default to false), things work fine. Then if I switch to DELAY_DEQUEUE at runtime, things start using a lot more power.

The interesting bit is if I switch to NO_DELAY_DEQUEUE again at runtime, things don't go back to normal. Rather they stay the same, using a lot more energy.

I wonder if we're leaving some unbalanced state somewhere while DELAY_DEQUEUE is on, something that is signalling we have more load/utilization than we actually do.

The PELT signals look reasonable from what I can see. We don't seem to be boosting frequencies, but we're running things mostly on big cores with DELAY_DEQUEUE on.

I'll keep investigating this. Please let me know if you need some additional data or testing and I can get that going.

author maintainer
#56 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 5 lines · Luis Machado

Booting the kernel with NO_DELAY_DEQUEUE (default to false), things work fine. Then if I switch to DELAY_DEQUEUE at runtime, things start using a lot more power.

The interesting bit is if I switch to NO_DELAY_DEQUEUE again at runtime, things don't go back to normal. Rather they stay the same, using a lot more energy.

Ooh, cute.. weird. I'll try and see if we leak state somehow.

#57 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Luis Machado

Booting the kernel with NO_DELAY_DEQUEUE (default to false), things work fine. Then if I switch to DELAY_DEQUEUE at runtime, things start using a lot more power.

The interesting bit is if I switch to NO_DELAY_DEQUEUE again at runtime, things don't go back to normal. Rather they stay the same, using a lot more energy.

Ooh, cute.. weird. I'll try and see if we leak state somehow.

Hi. I'm working on uclamp anyway so I just gave this series (and all fixes mentioned in this thread) a go.

It seems after running with this series for a while, suddenly uclamp_max no longer works. Running a task with uclamp_max of 110 still sees an rq uclamp_max of 1024. Also weirdly, the PELT signal of that uclamp_max is completely broken. It goes up to a very high value and then suddenly drops to almost 0 from time to time.

I wonder if the reason Luis saw that is because he was the first one to run delayed dequeue with uclamp. I'm not familiar enough with the series to know how it could possibly affect uclamp so investigating in this front may be worth trying. Maybe some leaked uclamp state on the rq?

#58 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra
Quoted 9 lines · Peter Zijlstra
Quoted 7 lines · Luis Machado
Quoted 6 lines

Bisecting through the patches in this series, I ended up with patch 08/10 as the one that improved things overall for these benchmarks.

I'd like to investigate this further to understand the reason behind some of these dramatic improvements.

Quoted 7 lines
Quoted 4 lines

Investigating these improvements a bit more, I noticed they came with a significantly higher power usage on the Pixel6 (where EAS is enabled). I bisected it down to the delayed dequeue patch. Disabling DELAY_DEQUEUE and DELAY_ZERO at runtime doesn't help in bringing the power usage down.

Hmm, that is unexpected. The intent was for NO_DELAY_DEQUEUE to fully disable things. I'll go have a prod at it.

I'm running a few more numbers to confirm this situation with the feature switches.

Quoted 9 lines
Quoted 6 lines

Though I don't fully understand the reason behind this change in behavior yet, I did spot the benchmark processes running almost entirely on the big core cluster, with little to no use of the little core and mid core clusters.

That would explain higher power usage and also the significant jump in performance.

ISTR you (arm) has these tools to trace and plot the varioud util values. This should be readily reflected there if that is the case, no?

Indeed we do, but I'm still in the process of compiling those numbers into a meaningful plot, so I'm afraid I don't have those handy yet, sorry.

Quoted 10 lines
Quoted 4 lines

I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of utilization/load of the runqueue and, as a consequence, we're scheduling things to run on higher OPP's in the big cores, leading to poor decisions for energy efficiency.

Notably util_est_update() gets delayed. Given we don't actually do an enqueue when a delayed task gets woken, it didn't seem to make sense to update that sooner.

I'll go over all that again

#59 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra

(removes apparently busted bytedance.com address and retries xmit)

Greetings!

With this version, the better CPU distribution (for tbench synchronous net-blasting) closed the CFS vs EEVDF throughput deficit. I verified both by rolling the previous version forward and back-porting to 6.1 where I've got CFS and EEVDF to re-compare, now with both dequeue delay patch versions.

As usual, there will be winners and losers, but (modulo dead buglet) it looks kinda promising to me.

Distribution of single pinned buddy pair measured in master:
DELAY_DEQUEUE
----------------------------------------------------------------------------------------------------------
Task | Runtime ms | Switches | Avg delay ms | Max delay ms | Sum delay ms |
----------------------------------------------------------------------------------------------------------

  tbench:(2)            |   6277.099 ms |  1597104 | avg:   0.003 ms | max:   0.129 ms | sum: 4334.723 ms |
  tbench_srv:(2)        |   5724.971 ms |  1682629 | avg:   0.001 ms | max:   0.083 ms | sum: 2076.616 ms |

----------------------------------------------------------------------------------------------------------
TOTAL: | 12021.128 ms | 3280275 | | 1.729 ms | 6425.483 ms |
----------------------------------------------------------------------------------------------------------
client/server CPU distribution ~52%/48%

NO_DELAY_DEQUEUE
----------------------------------------------------------------------------------------------------------
Task | Runtime ms | Switches | Avg delay ms | Max delay ms | Sum delay ms |
----------------------------------------------------------------------------------------------------------

  tbench:(2)            |   6724.774 ms |  1546761 | avg:   0.002 ms | max:   0.409 ms | sum: 2443.549 ms |
  tbench_srv:(2)        |   5275.329 ms |  1571688 | avg:   0.002 ms | max:   0.086 ms | sum: 2734.151 ms |

----------------------------------------------------------------------------------------------------------
TOTAL: | 12019.641 ms | 3119000 | | 9996.367 ms | 15187.833 ms |
----------------------------------------------------------------------------------------------------------
client/server CPU distribution ~56%/44%

Note switches and delay sum. For tbench, they translate directly to throughput. The other shoe lands with async CPU hog net-blasters, for those, scheduler cycles tends to be wasted cycles.

-Mike

#60 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

In reply to Peter Zijlstra
Quoted 15 lines · Peter Zijlstra

Extend / fix 86bfbb7ce4f6 ("sched/fair: Add lag based placement") by noting that lag is fundamentally a temporal measure. It should not be carried around indefinitely.

OTOH it should also not be instantly discarded, doing so will allow a task to game the system by purposefully (micro) sleeping at the end of its time quantum.

Since lag is intimately tied to the virtual time base, a wall-time based decay is also insufficient, notably competition is required for any of this to make sense.

Instead, delay the dequeue and keep the 'tasks' on the runqueue, competing until they are eligible.

Quoted 7 lines

Strictly speaking, we only care about keeping them until the 0-lag point, but that is a difficult proposition, instead carry them around until they get picked again, and dequeue them at that point.

Since we should have dequeued them at the 0-lag point, truncate lag (eg. don't let them earn positive lag).

I stumbled upon a subtle change between CFS and EEVDF which kinda relates to the question how long lag should be carried around, but on the other side of the 0-lag value.

Assume that we have two tasks taking turns on a single CPU. Task 1 does something and wakes up Task 2.
Task 2 does something and goes to sleep.
And we're just repeating that.
Task 1 and task 2 only run for very short amounts of time, i.e. much shorter than a regular time slice.

Let's further assume, that task 1 runs longer than task 2.
In CFS, this means, that vruntime of task 1 starts to outrun the vruntime of task 2. This means that vruntime(task2) < vruntime(task1). Hence, task 2 always gets picked on wake up because it has the smaller vruntime. In EEVDF, this would translate to a permanent positive lag, which also causes task 2 to get consistently scheduled on wake up.

Let's now assume, that ocassionally, task 2 runs a little bit longer than task 1. In CFS, this means, that task 2 can close the vruntime gap by a bit, but, it can easily remain below the value of task 1. Task 2 would still get picked on wake up.
With EEVDF, in its current form, task 2 will now get a negative lag, which in turn, will cause it not being picked on the next wake up.

So, it seems we have a change in the level of how far the both variants look into the past. CFS being willing to take more history into account, whereas EEVDF does not (with update_entity_lag setting the lag value from scratch, and place_entity not taking the original vruntime into account).

All of this can be seen as correct by design, a task consumes more time than the others, so it has to give way to others. The big difference is now, that CFS allowed a task to collect some bonus by constantly using less CPU time than others and trading that time against ocassionally taking more CPU time. EEVDF could do the same thing, by allowing the accumulation of (positive) lag, which can then be traded against the one time the task would get negative lag.

This might of course clash with other EEVDF assumptions but blends with the question on how long lag should be preserved.

I stumbled upon a performance degredation caused by this nuance. EEVDF is not necessarily at fault here. The problem rather lies in a component that relies on CFS allowing the above mentioned bonus aggregation while knowing that the corresponding task 2 has to run.
See: https://lore.kernel.org/netdev/ZWbapeL34Z8AMR5f@DESKTOP-2CCOB1S./T/
With the reasoning above, the degradation can be explained perfectly, and even be fixed by allowing the accumulation of lag by applying the patch below.

I was just wondering if my assumptions here are correct and whether this would be an implicit change that one should be aware about.

kernel/sched/fair.c +1 -1
index 03be0d1330a6..b83a72311d2a 100644
@@ -701,7 +701,7 @@ static void update_entity_lag(struct cfs_rq *cfs_rq, struct sched_entity *se)
        s64 lag, limit;
 
        SCHED_WARN_ON(!se->on_rq);
-       lag = avg_vruntime(cfs_rq) - se->vruntime;
+       lag = se->vlag + avg_vruntime(cfs_rq) - se->vruntime;
 
        limit = calc_delta_fair(max_t(u64, 2*se->slice, TICK_NSEC), se);
        se->vlag = clamp(lag, -limit, limit);

A message that is not in the archive (sent off-list or not yet received): [email protected]

author maintainer
#62 lore raw

Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue

Quoted 6 lines · Libo Chen

Hi Peter,

We are observing massive hackbench regressions with this patchset as well as the newer version from your queue. I don’t know if you are still working on complete EEVDF, just want to bring this issue to your attention if it’s still in play.

Yeha, I'm still (slowly) poking at this. I was planning to post a new set in a week or so.

Quoted 9 lines

The test system is a two-socket Zen4 EYPC with total of 192C/384T, intel does suffer too but to a lesser degree. Kernel is v6.10 based off the latest schedule/core. From 48 groups to 296 groups, process/threads via socket degrades <= 60%, process/threads via pipe can have regression at whopping 110% or more.

It turns out in deactivate_task(), this patchset changes the order of dequeue_task() and setting p->on_rq such that p->on_rq is set after dequeue_task() gets called.

Bah, I knew I had to double check that,.. but then I never did :/

Quoted 10 lines

My understanding is TASK_ON_RQ_MIGRATING
allows src rq lock to be released to other tasks since soon-to-be-dequeued task is migrating anyway, so holding two rq locks while dequeuing seems to be a quite horrible thing to do. The RFC patch is a bit tricky to get the order back, but a fix can be easily done to your newer version specifically to “sched: Split DEQUEUE_SLEEP from deactivate_task()” like below:

void deactivate_task(struct rq *rq, struct task_struct *p, int flags) {

Review

Quoted 7 lines
(quoted diff) +2 -2
-       dequeue_task(rq, p, flags);
-
        WRITE_ONCE(p->on_rq, TASK_ON_RQ_MIGRATING);
        ASSERT_EXCLUSIVE_WRITER(p->on_rq);
+
+       dequeue_task(rq, p, flags);
 }

Thanks, I'll fold that in and try and put your name somewhere.

author maintainer
#63 lore raw

RFC PATCH 9/10 sched/eevdf: Allow shorter slices to wakeup-preempt

In reply to Peter Zijlstra

Part of the reason to have shorter slices is to improve responsiveness. Allow shorter slices to preempt longer slices on wakeup.

Task | Runtime ms | Switches | Avg delay ms | Max delay ms | Sum delay ms |

100ms massive_intr 500us cyclictest NO_PREEMPT_SHORT

  1 massive_intr:(5)      | 846018.956 ms |   779188 | avg:   0.273 ms | max:  58.337 ms | sum:212545.245 ms |
  2 massive_intr:(5)      | 853450.693 ms |   792269 | avg:   0.275 ms | max:  71.193 ms | sum:218263.588 ms |
  3 massive_intr:(5)      | 843888.920 ms |   771456 | avg:   0.277 ms | max:  92.405 ms | sum:213353.221 ms |
  1 chromium-browse:(8)   |  53015.889 ms |   131766 | avg:   0.463 ms | max:  36.341 ms | sum:60959.230  ms |
  2 chromium-browse:(8)   |  53864.088 ms |   136962 | avg:   0.480 ms | max:  27.091 ms | sum:65687.681  ms |
  3 chromium-browse:(9)   |  53637.904 ms |   132637 | avg:   0.481 ms | max:  24.756 ms | sum:63781.673  ms |
  1 cyclictest:(5)        |  12615.604 ms |   639689 | avg:   0.471 ms | max:  32.272 ms | sum:301351.094 ms |
  2 cyclictest:(5)        |  12511.583 ms |   642578 | avg:   0.448 ms | max:  44.243 ms | sum:287632.830 ms |
  3 cyclictest:(5)        |  12545.867 ms |   635953 | avg:   0.475 ms | max:  25.530 ms | sum:302374.658 ms |

100ms massive_intr 500us cyclictest PREEMPT_SHORT

  1 massive_intr:(5)      | 839843.919 ms |   837384 | avg:   0.264 ms | max:  74.366 ms | sum:221476.885 ms |
  2 massive_intr:(5)      | 852449.913 ms |   845086 | avg:   0.252 ms | max:  68.162 ms | sum:212595.968 ms |
  3 massive_intr:(5)      | 839180.725 ms |   836883 | avg:   0.266 ms | max:  69.742 ms | sum:222812.038 ms |
  1 chromium-browse:(11)  |  54591.481 ms |   138388 | avg:   0.458 ms | max:  35.427 ms | sum:63401.508  ms |
  2 chromium-browse:(8)   |  52034.541 ms |   132276 | avg:   0.436 ms | max:  31.826 ms | sum:57732.958  ms |
  3 chromium-browse:(8)   |  55231.771 ms |   141892 | avg:   0.469 ms | max:  27.607 ms | sum:66538.697  ms |
  1 cyclictest:(5)        |  13156.391 ms |   667412 | avg:   0.373 ms | max:  38.247 ms | sum:249174.502 ms |
  2 cyclictest:(5)        |  12688.939 ms |   665144 | avg:   0.374 ms | max:  33.548 ms | sum:248509.392 ms |
  3 cyclictest:(5)        |  13475.623 ms |   669110 | avg:   0.370 ms | max:  37.819 ms | sum:247673.390 ms |

As per the numbers the, this makes cyclictest (short slice) it's max-delay more consistent and consistency drops the sum-delay. The trade-off is that the massive_intr (long slice) gets more context switches and a slight increase in sum-delay.

---
 kernel/sched/fair.c     |   11 ++++++++---
 kernel/sched/features.h |    4 ++++
 2 files changed, 12 insertions(+), 3 deletions(-)
kernel/sched/fair.c +8 -3
@@ -8538,9 +8538,14 @@ static void check_preempt_wakeup_fair(st
 	cfs_rq = cfs_rq_of(se);
 	update_curr(cfs_rq);
 
-	/*
-	 * XXX pick_eevdf(cfs_rq) != se ?
-	 */
+	if (sched_feat(PREEMPT_SHORT) && pse->slice < se->slice &&
+	    entity_eligible(cfs_rq, pse) &&
+	    (s64)(pse->deadline - se->deadline) < 0 &&
+	    se->vlag == se->deadline) {
+		/* negate RUN_TO_PARITY */
+		se->vlag = se->deadline - 1;
+	}
+
 	if (pick_eevdf(cfs_rq) == pse)
 		goto preempt;
 
kernel/sched/features.h +4 -0
@@ -14,6 +14,10 @@ SCHED_FEAT(PLACE_DEADLINE_INITIAL, true)
  * 0-lag point or until is has exhausted it's slice.
  */
 SCHED_FEAT(RUN_TO_PARITY, true)
+/*
+ * Allow tasks with a shorter slice to disregard RUN_TO_PARITY
+ */
+SCHED_FEAT(PREEMPT_SHORT, true)
 
 /*
  * Prefer to schedule the task we woke last (assuming it failed
author maintainer
#64 lore raw

RFC PATCH 10/10 sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

In reply to Peter Zijlstra

Allow applications to directly set a suggested request/slice length using sched_attr::sched_runtime.

The implementation clamps the value to: 0.1[ms] <= slice <= 100[ms] which is 1/10 the size of HZ=1000 and 10 times the size of HZ=100.

Applications should strive to use their periodic runtime at a high confidence interval (95%+) as the target slice. Using a smaller slice will introduce undue preemptions, while using a larger value will increase latency.

For all the following examples assume a scheduling quantum of 8, and for consistency all examples have W=4:

{A,B,C,D}(w=1,r=8):

ABCD...
+---+---+---+---

t=0, V=1.5 t=1, V=3.5
A |------< A |------<
B |------< B |------<
C |------< C |------<
D |------< D |------<
---+*------+-------+--- ---+--*----+-------+---

t=2, V=5.5 t=3, V=7.5
A |------< A |------<
B |------< B |------<
C |------< C |------<
D |------< D |------<
---+----*--+-------+--- ---+------*+-------+---

Note: 4 identical tasks in FIFO order

~~~

{A,B}(w=1,r=16) C(w=2,r=16)

AACCBBCC...
+---+---+---+---

t=0, V=1.25 t=2, V=5.25

  A  |--------------<                   A                  |--------------<
  B   |--------------<                  B   |--------------<
  C    |------<                         C    |------<
  ---+*------+-------+---               ---+----*--+-------+---

t=4, V=8.25 t=6, V=12.25

  A                  |--------------<   A                  |--------------<
  B   |--------------<                  B                   |--------------<
  C            |------<                 C            |------<
  ---+-------*-------+---               ---+-------+---*---+---

Note: 1 heavy task -- because q=8, double r such that the deadline of the w=2
task doesn't go below q.

Note: observe the full schedule becomes: W*max(r_i/w_i) = 4*2q = 8q in length.

Note: the period of the heavy task is half the full period at:

      W*(r_i/w_i) = 4*(2q/2) = 4q

~~~

{A,C,D}(w=1,r=16) B(w=1,r=8):

BAACCBDD...
+---+---+---+---

t=0, V=1.5 t=1, V=3.5
A |--------------< A |---------------<
B |------< B |------<
C |--------------< C |--------------<
D |--------------< D |--------------<
---+*------+-------+--- ---+--*----+-------+---

t=3, V=7.5 t=5, V=11.5

  A                  |---------------<  A                  |---------------<
  B           |------<                  B           |------<
  C    |--------------<                 C                    |--------------<
  D     |--------------<                D     |--------------<
  ---+------*+-------+---               ---+-------+--*----+---

t=6, V=13.5
A |---------------<
B |------<
C |--------------<
D |--------------<
---+-------+----*--+---

Note: 1 short task -- again double r so that the deadline of the short task
won't be below q. Made B short because its not the leftmost task, but is eligible with the 0,1,2,3 spread.

Note: like with the heavy task, the period of the short task observes:

      W*(r_i/w_i) = 4*(1q/1) = 4q

~~~

A(w=1,r=16) B(w=1,r=8) C(w=2,r=16)

BCCAABCC...
+---+---+---+---

t=0, V=1.25 t=1, V=3.25

  A  |--------------<                   A  |--------------<
  B   |------<                          B           |------<
  C    |------<                         C    |------<
  ---+*------+-------+---               ---+--*----+-------+---

t=3, V=7.25 t=5, V=11.25

  A  |--------------<                   A                  |--------------<
  B           |------<                  B           |------<
  C            |------<                 C            |------<
  ---+------*+-------+---               ---+-------+--*----+---

t=6, V=13.25
A |--------------<
B |------<
C |------<
---+-------+----*--+---

Note: 1 heavy and 1 short task -- combine them all.

Note: both the short and heavy task end up with a period of 4q

~~~

A(w=1,r=16) B(w=2,r=16) C(w=1,r=8)

BBCAABBC...
+---+---+---+---

t=0, V=1 t=2, V=5

  A  |--------------<                   A  |--------------<
  B   |------<                          B           |------<
  C    |------<                         C    |------<
  ---+*------+-------+---               ---+----*--+-------+---

t=3, V=7 t=5, V=11

  A  |--------------<                   A                  |--------------<
  B           |------<                  B           |------<
  C            |------<                 C            |------<
  ---+------*+-------+---               ---+-------+--*----+---

t=7, V=15
A |--------------<
B |------<
C |------<
---+-------+------*+---

Note: as before but permuted

~~~

Quoted 1 line

From all this it can be deduced that, for the steady state:

- the total period (P) of a schedule is: W*max(r_i/w_i)
- the average period of a task is: W*(r_i/w_i)
- each task obtains the fair share: w_i/W of each full period P

---
 include/linux/sched.h |    5 ++++-
 kernel/sched/core.c   |   33 ++++++++++++++++++++++++++-------
 kernel/sched/debug.c  |    3 ++-
 kernel/sched/fair.c   |    6 ++++--
 4 files changed, 36 insertions(+), 11 deletions(-)
include/linux/sched.h +4 -1
@@ -542,7 +542,10 @@ struct sched_entity {
 
 	struct list_head		group_node;
 	unsigned int			on_rq;
-	unsigned int			sched_delayed;
+
+	unsigned char			sched_delayed;
+	unsigned char			custom_slice;
+					/* 2 byte hole */
 
 	u64				exec_start;
 	u64				sum_exec_runtime;
kernel/sched/core.c +26 -7
@@ -4550,7 +4550,6 @@ static void __sched_fork(unsigned long c
 	p->se.nr_migrations		= 0;
 	p->se.vruntime			= 0;
 	p->se.vlag			= 0;
-	p->se.slice			= sysctl_sched_base_slice;
 	INIT_LIST_HEAD(&p->se.group_node);
 
 #ifdef CONFIG_FAIR_GROUP_SCHED
@@ -4801,6 +4800,8 @@ int sched_fork(unsigned long clone_flags
 
 		p->prio = p->normal_prio = p->static_prio;
 		set_load_weight(p, false);
+		p->se.custom_slice = 0;
+		p->se.slice = sysctl_sched_base_slice;
 
 		/*
 		 * We don't need the reset flag anymore after the fork. It has
@@ -7627,10 +7628,20 @@ static void __setscheduler_params(struct
 
 	p->policy = policy;
 
-	if (dl_policy(policy))
+	if (dl_policy(policy)) {
 		__setparam_dl(p, attr);
-	else if (fair_policy(policy))
+	} else if (fair_policy(policy)) {
 		p->static_prio = NICE_TO_PRIO(attr->sched_nice);
+		if (attr->sched_runtime) {
+			p->se.custom_slice = 1;
+			p->se.slice = clamp_t(u64, attr->sched_runtime,
+					      NSEC_PER_MSEC/10,   /* HZ=1000 * 10 */
+					      NSEC_PER_MSEC*100); /* HZ=100  / 10 */
+		} else {
+			p->se.custom_slice = 0;
+			p->se.slice = sysctl_sched_base_slice;
+		}
+	}
 
 	/*
 	 * __sched_setscheduler() ensures attr->sched_priority == 0 when
@@ -7812,7 +7823,9 @@ static int __sched_setscheduler(struct t
 	 * but store a possible modification of reset_on_fork.
 	 */
 	if (unlikely(policy == p->policy)) {
-		if (fair_policy(policy) && attr->sched_nice != task_nice(p))
+		if (fair_policy(policy) &&
+		    (attr->sched_nice != task_nice(p) ||
+		     (attr->sched_runtime && attr->sched_runtime != p->se.slice)))
 			goto change;
 		if (rt_policy(policy) && attr->sched_priority != p->rt_priority)
 			goto change;
@@ -7958,6 +7971,9 @@ static int _sched_setscheduler(struct ta
 		.sched_nice	= PRIO_TO_NICE(p->static_prio),
 	};
 
+	if (p->se.custom_slice)
+		attr.sched_runtime = p->se.slice;
+
 	/* Fixup the legacy SCHED_RESET_ON_FORK hack. */
 	if ((policy != SETPARAM_POLICY) && (policy & SCHED_RESET_ON_FORK)) {
 		attr.sched_flags |= SCHED_FLAG_RESET_ON_FORK;
@@ -8124,12 +8140,14 @@ static int sched_copy_attr(struct sched_
 
 static void get_params(struct task_struct *p, struct sched_attr *attr)
 {
-	if (task_has_dl_policy(p))
+	if (task_has_dl_policy(p)) {
 		__getparam_dl(p, attr);
-	else if (task_has_rt_policy(p))
+	} else if (task_has_rt_policy(p)) {
 		attr->sched_priority = p->rt_priority;
-	else
+	} else {
 		attr->sched_nice = task_nice(p);
+		attr->sched_runtime = p->se.slice;
+	}
 }
 
 /**
@@ -10084,6 +10102,7 @@ void __init sched_init(void)
 	}
 
 	set_load_weight(&init_task, false);
+	init_task.se.slice = sysctl_sched_base_slice,
 
 	/*
 	 * The boot idle thread does lazy MMU switching as well:
kernel/sched/debug.c +2 -1
@@ -579,11 +579,12 @@ print_task(struct seq_file *m, struct rq
 	else
 		SEQ_printf(m, " %c", task_state_to_char(p));
 
-	SEQ_printf(m, "%15s %5d %9Ld.%06ld %c %9Ld.%06ld %9Ld.%06ld %9Ld.%06ld %9Ld %5d ",
+	SEQ_printf(m, "%15s %5d %9Ld.%06ld %c %9Ld.%06ld %c %9Ld.%06ld %9Ld.%06ld %9Ld %5d ",
 		p->comm, task_pid_nr(p),
 		SPLIT_NS(p->se.vruntime),
 		entity_eligible(cfs_rq_of(&p->se), &p->se) ? 'E' : 'N',
 		SPLIT_NS(p->se.deadline),
+		p->se.custom_slice ? 'S' : ' ',
 		SPLIT_NS(p->se.slice),
 		SPLIT_NS(p->se.sum_exec_runtime),
 		(long long)(p->nvcsw + p->nivcsw),
kernel/sched/fair.c +4 -2
@@ -984,7 +984,8 @@ static void update_deadline(struct cfs_r
 	 * nice) while the request time r_i is determined by
 	 * sysctl_sched_base_slice.
 	 */
-	se->slice = sysctl_sched_base_slice;
+	if (!se->custom_slice)
+		se->slice = sysctl_sched_base_slice;
 
 	/*
 	 * EEVDF: vd_i = ve_i + r_i / w_i
@@ -5169,7 +5170,8 @@ place_entity(struct cfs_rq *cfs_rq, stru
 	u64 vslice, vruntime = avg_vruntime(cfs_rq);
 	s64 lag = 0;
 
-	se->slice = sysctl_sched_base_slice;
+	if (!se->custom_slice)
+		se->slice = sysctl_sched_base_slice;
 	vslice = calc_delta_fair(se->slice, se);
 
 	/*
#65 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

On Fri, 05 Apr 2024 12:28:04 +0200 Peter Zijlstra <[email protected]>

Quoted 6 lines

Allow applications to directly set a suggested request/slice length using sched_attr::sched_runtime.

The implementation clamps the value to: 0.1[ms] <= slice <= 100[ms] which is 1/10 the size of HZ=1000 and 10 times the size of HZ=100.

Given HZ=100 for example, what is preventing applications of suggested slice=0.5ms from running 5ms a tick? If slice is 90ms otoh, is tick able to kick the curr that has been on cpu for 10ms off cpu, given cfs_rq->nr_running > 1?

Quoted 12 lines
kernel/sched/fair.c +2 -1
@@ -984,7 +984,8 @@ static void update_deadline(struct cfs_r
 	 * nice) while the request time r_i is determined by
 	 * sysctl_sched_base_slice.
 	 */
-	se->slice = sysctl_sched_base_slice;
+	if (!se->custom_slice)
+		se->slice = sysctl_sched_base_slice;
 
 	/*
 	 * EEVDF: vd_i = ve_i + r_i / w_i
#66 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

In reply to Peter Zijlstra

kernel/sched/core.c static int __sched_setscheduler(struct t · line 7829

Quoted 12 lines · Peter Zijlstra
kernel/sched/core.c +3 -1
@@ -7812,7 +7823,9 @@ static int __sched_setscheduler(struct t
          * but store a possible modification of reset_on_fork.
          */
         if (unlikely(policy == p->policy)) {
-               if (fair_policy(policy) && attr->sched_nice != task_nice(p))
+               if (fair_policy(policy) &&
+                   (attr->sched_nice != task_nice(p) ||
+                    (attr->sched_runtime && attr->sched_runtime != p->se.slice)))
                         goto change;

Can we make that only look at attr->sched_runtime != p->se.slice? Doing so lets you clear a custom slice by.. clearing it.. rather than making tools rummage about for the proper value to overwrite.

-Mike

author maintainer
#67 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

Quoted 17 lines · Mike Galbraith

kernel/sched/core.c static int __sched_setscheduler(struct t · line 7829

Quoted 12 lines · Peter Zijlstra
kernel/sched/core.c +3 -1
@@ -7812,7 +7823,9 @@ static int __sched_setscheduler(struct t
          * but store a possible modification of reset_on_fork.
          */
         if (unlikely(policy == p->policy)) {
-               if (fair_policy(policy) && attr->sched_nice != task_nice(p))
+               if (fair_policy(policy) &&
+                   (attr->sched_nice != task_nice(p) ||
+                    (attr->sched_runtime && attr->sched_runtime != p->se.slice)))
                         goto change;

Can we make that only look at attr->sched_runtime != p->se.slice? Doing so lets you clear a custom slice by.. clearing it.. rather than making tools rummage about for the proper value to overwrite.

Duh yes. Seems I already did the right thing in __setscheduler_params() for that.

Done.

#68 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

In reply to Peter Zijlstra
Quoted 6 lines · Peter Zijlstra

Allow applications to directly set a suggested request/slice length using sched_attr::sched_runtime.

The implementation clamps the value to: 0.1[ms] <= slice <= 100[ms] which is 1/10 the size of HZ=1000 and 10 times the size of HZ=100.

Quoted 7 lines

Applications should strive to use their periodic runtime at a high confidence interval (95%+) as the target slice. Using a smaller slice will introduce undue preemptions, while using a larger value will increase latency.

Is it possible to leverage this task slice to do better task wakeup placement? The idea is that, the smaller the slice the wakee has, the less idle CPU it should scan. This can reduce wake latency and inhibit costly task migration, especially on large systems.

We did some experiments and got some performance improvements:

From 9cb806476586d7048fcbd0f66d0101f0dbb8fd2b Mon Sep 17 00:00:00 2001
From: Chen Yu <[email protected]>
Date: Tue, 7 May 2024 22:36:29 +0800
Subject: [RFC PATCH] sched/eevdf: Use customized slice to reduce wakeup latency
and inhibit task migration

Problem 1:
The overhead of task migration is high on many-core system. The overhead brings performance penalty due to broken cache locality/higher cache-to-cache latency.

Problem 2:
During wakeup, the time spent on searching for an idle CPU is costly on many-core system. Besides, access to other CPU's rq statistics brings cace contention:

available_idle_cpu(cpu) -> idle_cpu(cpu) -> {rq->curr, rq->nr_running}

Although SIS_UTIL throttles the scan depth based on system utilization, there is requirement to further limit the scan depth for specific workload, especially for short duration wakee.

Now we have the interface to customize the request/slice. The smaller the slice is, the earlier the task can be picked up, and the lower wakeup latency the task expects. Leverage the wakee's slice to further throttle the idle CPU scan depth - the shorter slice, the less CPUs to scan.

Test on 240 CPUs, 2 sockets system. With SNC(sub-numa-cluster) enabled, each LLC domain has 60 CPUs. There is noticeable improvement of netperf. (With SNC disabled, more improvements should be seen because C2C is higher)

The global slice is 3 msec(sysctl_sched_base_slice) by default on my ubuntu 22.04, and the customized slice is set to 0.1 msec for both netperf and netserver:

for i in $(seq 1 $job); do

	netperf_slice -e 100000 -4 -H 127.0.01 -t TCP_RR -c -C -l 100 &

done

case            	load    	baseline(std%)	compare%( std%)
TCP_RR          	60-threads	 1.00 (  1.60)	 +0.35 (  1.73)
TCP_RR          	120-threads	 1.00 (  1.34)	 -0.96 (  1.24)
TCP_RR          	180-threads	 1.00 (  1.59)	+92.20 (  4.24)
TCP_RR          	240-threads	 1.00 (  9.71)	+43.11 (  2.97)
---
 kernel/sched/fair.c     | 23 ++++++++++++++++++++---
 kernel/sched/features.h |  1 +
 2 files changed, 21 insertions(+), 3 deletions(-)
kernel/sched/fair.c +20 -3
index edc23f6588a3..f269ae7d6e24 100644
@@ -7368,6 +7368,24 @@ static inline int select_idle_smt(struct task_struct *p, struct sched_domain *sd
 
 #endif /* CONFIG_SCHED_SMT */
 
+/*
+ * Scale the scan number of idle CPUs according to customized
+ * wakee's slice. The smaller the slice is, the earlier the task
+ * wants be picked up, thus the lower wakeup latency the task expects.
+ * The baseline is the global sysctl_sched_base_slice. Task slice
+ * smaller than the global one would shrink the scan number.
+ */
+static int adjust_idle_scan(struct task_struct *p, int nr)
+{
+	if (!sched_feat(SIS_FAST))
+		return nr;
+
+	if (!p->se.custom_slice || p->se.slice >= sysctl_sched_base_slice)
+		return nr;
+
+	return div_u64(nr * p->se.slice, sysctl_sched_base_slice);
+}
+
 /*
  * Scan the LLC domain for idle CPUs; this is dynamically regulated by
  * comparing the average scan cost (tracked in sd->avg_scan_cost) against the
@@ -7384,10 +7402,9 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
 	if (sched_feat(SIS_UTIL)) {
 		sd_share = rcu_dereference(per_cpu(sd_llc_shared, target));
 		if (sd_share) {
-			/* because !--nr is the condition to stop scan */
-			nr = READ_ONCE(sd_share->nr_idle_scan) + 1;
+			nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan));
 			/* overloaded LLC is unlikely to have idle cpu/core */
-			if (nr == 1)
+			if (nr <= 0)
 				return -1;
 		}
 	}
kernel/sched/features.h +1 -0
index 143f55df890b..176324236018 100644
@@ -50,6 +50,7 @@ SCHED_FEAT(TTWU_QUEUE, true)
  * When doing wakeups, attempt to limit superfluous scans of the LLC domain.
  */
 SCHED_FEAT(SIS_UTIL, true)
+SCHED_FEAT(SIS_FAST, true)
 
 /*
  * Issue a WARN when we do multiple update_rq_clock() calls
Signature · 1 line
2.25.1
#69 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

Quoted 12 lines · Chen Yu
Quoted 11 lines · Peter Zijlstra

Allow applications to directly set a suggested request/slice length using sched_attr::sched_runtime.

The implementation clamps the value to: 0.1[ms] <= slice <= 100[ms] which is 1/10 the size of HZ=1000 and 10 times the size of HZ=100.

Applications should strive to use their periodic runtime at a high confidence interval (95%+) as the target slice. Using a smaller slice will introduce undue preemptions, while using a larger value will increase latency.

Quoted 4 lines
Quoted 2 lines

Is it possible to leverage this task slice to do better task wakeup placement?

Slice being unrelated to placement makes its use in a placement related knob look wrong. Even the smallest possible slice is orders of magnitude larger than the cycle time of TCP_RR, making slice nearly irrelevant to the issue being demonstrating via TCP_RR. Even for that huge socket box, it won't take long as cycle time increases toward that smallest possible slice for the cost of needless wait to bury placement decision costs.

Quoted 3 lines

The idea is that, the smaller the slice the wakee has, the less idle CPU it should scan. This can reduce wake latency and inhibit costly task migration, especially on large systems.

Sure, this is an age old issue that's scaled up to size extra ugly in that huge socket box. Any solution needs to scale as well methinks, a simple fixed yardstick won't work, as the costs being mitigated vary wildly with platform size/shape.

-Mike

#70 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

Quoted 16 lines · Mike Galbraith
Quoted 15 lines · Chen Yu
Quoted 13 lines · Peter Zijlstra

Allow applications to directly set a suggested request/slice length using sched_attr::sched_runtime.

The implementation clamps the value to: 0.1[ms] <= slice <= 100[ms] which is 1/10 the size of HZ=1000 and 10 times the size of HZ=100.

Applications should strive to use their periodic runtime at a high confidence interval (95%+) as the target slice. Using a smaller slice will introduce undue preemptions, while using a larger value will increase latency.

Quoted 6 lines
Quoted 1 line

Is it possible to leverage this task slice to do better task wakeup placement?

Slice being unrelated to placement makes its use in a placement related knob look wrong. Even the smallest possible slice is orders of magnitude larger than the cycle time of TCP_RR, making slice nearly irrelevant to the issue being demonstrating via TCP_RR.

Yes, I agree that there is no direct relationship between the slice and the task placement. The idea is to use slice as an input hint from the user to tell the kernel how much latency this user cares(but not expect the task to last for that long).

Quoted 4 lines

Even for that huge socket box, it won't take long as cycle time increases toward that smallest possible slice for the cost of needless wait to bury placement decision costs.

I see. The wake up latency is composed of:
1. The time to do placement decision.
2. The time wait in the runqueue to be picked.

Even if the 1 has been reduced, the 2 might take more time if the runqueue is busy. We can mitigate this by checking if there is <= 1 short duration task on that target CPU.

Quoted 9 lines
Quoted 3 lines

The idea is that, the smaller the slice the wakee has, the less idle CPU it should scan. This can reduce wake latency and inhibit costly task migration, especially on large systems.

Sure, this is an age old issue that's scaled up to size extra ugly in that huge socket box. Any solution needs to scale as well methinks, a simple fixed yardstick won't work, as the costs being mitigated vary wildly with platform size/shape.

Understand, this slice based placement was chosen for experimental purpose, and seek for directions. And I agree we should take the platform size(such as CPU number) into consideration.

thanks,
Chenyu

#71 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

Quoted 2 lines · Chen Yu

And I agree we should take the platform size(such
as CPU number) into consideration.

I think you'll need more that that, because size agnostic, when it comes to latency, an idle CPU is damn hard to beat, making migration restrictions (traditional and obvious target) tend to leave highly annoying piles of collateral damage in their wake.

(spoken from BTDT perspective, have t-shirt, got butt kicked;)

-Mike

#72 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

In reply to Chen Yu

Hello Chenyu,

Quoted 104 lines · Chen Yu
Quoted 6 lines · Peter Zijlstra

Allow applications to directly set a suggested request/slice length using sched_attr::sched_runtime.

The implementation clamps the value to: 0.1[ms] <= slice <= 100[ms] which is 1/10 the size of HZ=1000 and 10 times the size of HZ=100.

Quoted 7 lines

Applications should strive to use their periodic runtime at a high confidence interval (95%+) as the target slice. Using a smaller slice will introduce undue preemptions, while using a larger value will increase latency.

Is it possible to leverage this task slice to do better task wakeup placement? The idea is that, the smaller the slice the wakee has, the less idle CPU it should scan. This can reduce wake latency and inhibit costly task migration, especially on large systems.

We did some experiments and got some performance improvements:

From 9cb806476586d7048fcbd0f66d0101f0dbb8fd2b Mon Sep 17 00:00:00 2001
From: Chen Yu <[email protected]>
Date: Tue, 7 May 2024 22:36:29 +0800
Subject: [RFC PATCH] sched/eevdf: Use customized slice to reduce wakeup latency
and inhibit task migration

Problem 1:
The overhead of task migration is high on many-core system. The overhead brings performance penalty due to broken cache locality/higher cache-to-cache latency.

Problem 2:
During wakeup, the time spent on searching for an idle CPU is costly on many-core system. Besides, access to other CPU's rq statistics brings cace contention:

available_idle_cpu(cpu) -> idle_cpu(cpu) -> {rq->curr, rq->nr_running}

Although SIS_UTIL throttles the scan depth based on system utilization, there is requirement to further limit the scan depth for specific workload, especially for short duration wakee.

Now we have the interface to customize the request/slice. The smaller the slice is, the earlier the task can be picked up, and the lower wakeup latency the task expects. Leverage the wakee's slice to further throttle the idle CPU scan depth - the shorter slice, the less CPUs to scan.

Test on 240 CPUs, 2 sockets system. With SNC(sub-numa-cluster) enabled, each LLC domain has 60 CPUs. There is noticeable improvement of netperf. (With SNC disabled, more improvements should be seen because C2C is higher)

The global slice is 3 msec(sysctl_sched_base_slice) by default on my ubuntu 22.04, and the customized slice is set to 0.1 msec for both netperf and netserver:

for i in $(seq 1 $job); do

	netperf_slice -e 100000 -4 -H 127.0.01 -t TCP_RR -c -C -l 100 &

done

case            	load    	baseline(std%)	compare%( std%)
TCP_RR          	60-threads	 1.00 (  1.60)	 +0.35 (  1.73)
TCP_RR          	120-threads	 1.00 (  1.34)	 -0.96 (  1.24)
TCP_RR          	180-threads	 1.00 (  1.59)	+92.20 (  4.24)
TCP_RR          	240-threads	 1.00 (  9.71)	+43.11 (  2.97)
---
 kernel/sched/fair.c     | 23 ++++++++++++++++++++---
 kernel/sched/features.h |  1 +
 2 files changed, 21 insertions(+), 3 deletions(-)
kernel/sched/fair.c +18 -0
index edc23f6588a3..f269ae7d6e24 100644
@@ -7368,6 +7368,24 @@ static inline int select_idle_smt(struct task_struct *p, struct sched_domain *sd
 
 #endif /* CONFIG_SCHED_SMT */
 
+/*
+ * Scale the scan number of idle CPUs according to customized
+ * wakee's slice. The smaller the slice is, the earlier the task
+ * wants be picked up, thus the lower wakeup latency the task expects.
+ * The baseline is the global sysctl_sched_base_slice. Task slice
+ * smaller than the global one would shrink the scan number.
+ */
+static int adjust_idle_scan(struct task_struct *p, int nr)
+{
+	if (!sched_feat(SIS_FAST))
+		return nr;
+
+	if (!p->se.custom_slice || p->se.slice >= sysctl_sched_base_slice)
+		return nr;
+
+	return div_u64(nr * p->se.slice, sysctl_sched_base_slice);
+}
+
 /*
  * Scan the LLC domain for idle CPUs; this is dynamically regulated by
  * comparing the average scan cost (tracked in sd->avg_scan_cost) against the

kernel/sched/fair.c static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool · line 7407

Quoted 8 lines
kernel/sched/fair.c +2 -2
@@ -7384,10 +7402,9 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
 	if (sched_feat(SIS_UTIL)) {
 		sd_share = rcu_dereference(per_cpu(sd_llc_shared, target));
 		if (sd_share) {
-			/* because !--nr is the condition to stop scan */> -			nr = READ_ONCE(sd_share->nr_idle_scan) + 1;
+			nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan));
 			/* overloaded LLC is unlikely to have idle cpu/core */
-			if (nr == 1)
+			if (nr <= 0)

I was wondering if this would preserve the current behavior with SIS_FAST toggled off? Since the implementation below still does a "--nr <= 0" , wouldn't it effectively visit one CPU less overall now?

Have you tried something similar to the below hunk?

	/* because !--nr is the condition to stop scan */
	nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan)) + 1;
	if (nr == 1)
		return -1;

I agree with Mike that looking at slice to limit scan-depth seems odd. My experience with netperf is that the workload cares more about the server-client being co-located on the closest cache domain and by limiting scan-depth using slice, this is indirectly achieved since all the wakeups carry the WF_SYNc flag.

P.S. have you tried using the slice in __select_idle_cpu()? Similar to sched_idle_cpu() check, perhaps an additional sched_preempt_short_cpu() which compares rq->curr->se.slice with the waking task's slice and returs that cpu if SIS_SHORT can help run the workload quicker? Note:
This will not work if the SIS scan itself is the largest overhead in the wakeup cycle and not the task placement itself. Previously during SIS_UTIL testing, to measure the overheads of scan vs placement, we would do a full scan but return the result that SIS_UTIL would have returned to determine the overhead of the search itself.

Quoted 8 lines
kernel/sched/fair.c +0 -0
 				return -1;
 		}
 	}
kernel/sched/features.h +0 -0
index 143f55df890b..176324236018 100644

kernel/sched/features.h SCHED_FEAT(TTWU_QUEUE, true) · line 56

Quoted 7 lines
kernel/sched/features.h +1 -0
@@ -50,6 +50,7 @@ SCHED_FEAT(TTWU_QUEUE, true)
  * When doing wakeups, attempt to limit superfluous scans of the LLC domain.
  */
 SCHED_FEAT(SIS_UTIL, true)
+SCHED_FEAT(SIS_FAST, true)
 
 /*
  * Issue a WARN when we do multiple update_rq_clock() calls

--
Thanks and Regards,
Prateek

#73 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

Hi Prateek,

Quoted 120 lines · K Prateek Nayak

Hello Chenyu,

Quoted 104 lines · Chen Yu
Quoted 6 lines · Peter Zijlstra

Allow applications to directly set a suggested request/slice length using sched_attr::sched_runtime.

The implementation clamps the value to: 0.1[ms] <= slice <= 100[ms] which is 1/10 the size of HZ=1000 and 10 times the size of HZ=100.

Quoted 7 lines

Applications should strive to use their periodic runtime at a high confidence interval (95%+) as the target slice. Using a smaller slice will introduce undue preemptions, while using a larger value will increase latency.

Is it possible to leverage this task slice to do better task wakeup placement? The idea is that, the smaller the slice the wakee has, the less idle CPU it should scan. This can reduce wake latency and inhibit costly task migration, especially on large systems.

We did some experiments and got some performance improvements:

From 9cb806476586d7048fcbd0f66d0101f0dbb8fd2b Mon Sep 17 00:00:00 2001
From: Chen Yu <[email protected]>
Date: Tue, 7 May 2024 22:36:29 +0800
Subject: [RFC PATCH] sched/eevdf: Use customized slice to reduce wakeup latency
and inhibit task migration

Problem 1:
The overhead of task migration is high on many-core system. The overhead brings performance penalty due to broken cache locality/higher cache-to-cache latency.

Problem 2:
During wakeup, the time spent on searching for an idle CPU is costly on many-core system. Besides, access to other CPU's rq statistics brings cace contention:

available_idle_cpu(cpu) -> idle_cpu(cpu) -> {rq->curr, rq->nr_running}

Although SIS_UTIL throttles the scan depth based on system utilization, there is requirement to further limit the scan depth for specific workload, especially for short duration wakee.

Now we have the interface to customize the request/slice. The smaller the slice is, the earlier the task can be picked up, and the lower wakeup latency the task expects. Leverage the wakee's slice to further throttle the idle CPU scan depth - the shorter slice, the less CPUs to scan.

Test on 240 CPUs, 2 sockets system. With SNC(sub-numa-cluster) enabled, each LLC domain has 60 CPUs. There is noticeable improvement of netperf. (With SNC disabled, more improvements should be seen because C2C is higher)

The global slice is 3 msec(sysctl_sched_base_slice) by default on my ubuntu 22.04, and the customized slice is set to 0.1 msec for both netperf and netserver:

for i in $(seq 1 $job); do

	netperf_slice -e 100000 -4 -H 127.0.01 -t TCP_RR -c -C -l 100 &

done

case            	load    	baseline(std%)	compare%( std%)
TCP_RR          	60-threads	 1.00 (  1.60)	 +0.35 (  1.73)
TCP_RR          	120-threads	 1.00 (  1.34)	 -0.96 (  1.24)
TCP_RR          	180-threads	 1.00 (  1.59)	+92.20 (  4.24)
TCP_RR          	240-threads	 1.00 (  9.71)	+43.11 (  2.97)
---
 kernel/sched/fair.c     | 23 ++++++++++++++++++++---
 kernel/sched/features.h |  1 +
 2 files changed, 21 insertions(+), 3 deletions(-)
kernel/sched/fair.c +18 -0
index edc23f6588a3..f269ae7d6e24 100644
@@ -7368,6 +7368,24 @@ static inline int select_idle_smt(struct task_struct *p, struct sched_domain *sd
 
 #endif /* CONFIG_SCHED_SMT */
 
+/*
+ * Scale the scan number of idle CPUs according to customized
+ * wakee's slice. The smaller the slice is, the earlier the task
+ * wants be picked up, thus the lower wakeup latency the task expects.
+ * The baseline is the global sysctl_sched_base_slice. Task slice
+ * smaller than the global one would shrink the scan number.
+ */
+static int adjust_idle_scan(struct task_struct *p, int nr)
+{
+	if (!sched_feat(SIS_FAST))
+		return nr;
+
+	if (!p->se.custom_slice || p->se.slice >= sysctl_sched_base_slice)
+		return nr;
+
+	return div_u64(nr * p->se.slice, sysctl_sched_base_slice);
+}
+
 /*
  * Scan the LLC domain for idle CPUs; this is dynamically regulated by
  * comparing the average scan cost (tracked in sd->avg_scan_cost) against the

kernel/sched/fair.c static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool · line 7407

Quoted 8 lines
kernel/sched/fair.c +2 -2
@@ -7384,10 +7402,9 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
 	if (sched_feat(SIS_UTIL)) {
 		sd_share = rcu_dereference(per_cpu(sd_llc_shared, target));
 		if (sd_share) {
-			/* because !--nr is the condition to stop scan */> -			nr = READ_ONCE(sd_share->nr_idle_scan) + 1;
+			nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan));
 			/* overloaded LLC is unlikely to have idle cpu/core */
-			if (nr == 1)
+			if (nr <= 0)

I was wondering if this would preserve the current behavior with SIS_FAST toggled off? Since the implementation below still does a "--nr <= 0" , wouldn't it effectively visit one CPU less overall now?

Quoted 7 lines

Have you tried something similar to the below hunk?

	/* because !--nr is the condition to stop scan */
	nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan)) + 1;
	if (nr == 1)
		return -1;

Yeah, right, to keep the scan depth consistent, the "+1" should be kept.

Quoted 6 lines

I agree with Mike that looking at slice to limit scan-depth seems odd. My experience with netperf is that the workload cares more about the server-client being co-located on the closest cache domain and by limiting scan-depth using slice, this is indirectly achieved since all the wakeups carry the WF_SYNc flag.

Exactly. This is the original motivation.

Quoted 4 lines

P.S. have you tried using the slice in __select_idle_cpu()? Similar to sched_idle_cpu() check, perhaps an additional sched_preempt_short_cpu() which compares rq->curr->se.slice with the waking task's slice and returs that cpu if SIS_SHORT can help run the workload quicker?

This is a good idea, it seems to be benefit PREEMPT_SHORT. If the customized task slice is introduced, we can leverage this hint for latency related optimization. Task wakeup is one thing, I can also think of other aspects, like idle load balance, etc. I'm not sure what is the proper usage of the task slice though, this is why I sent this RFC.

Quoted 7 lines

Note:
This will not work if the SIS scan itself is the largest overhead in the wakeup cycle and not the task placement itself. Previously during SIS_UTIL testing, to measure the overheads of scan vs placement, we would do a full scan but return the result that SIS_UTIL would have returned to determine the overhead of the search itself.

Regarding the task placement, do you mean the time between a task is enqueued and picked up? Do you have any recommendation which workload can expose the scan overhead most?

thanks,
Chenyu

Quoted 19 lines
Quoted 8 lines
kernel/sched/fair.c +0 -0
 				return -1;
 		}
 	}
kernel/sched/features.h +0 -0
index 143f55df890b..176324236018 100644

kernel/sched/features.h SCHED_FEAT(TTWU_QUEUE, true) · line 56

Quoted 7 lines
kernel/sched/features.h +1 -0
@@ -50,6 +50,7 @@ SCHED_FEAT(TTWU_QUEUE, true)
  * When doing wakeups, attempt to limit superfluous scans of the LLC domain.
  */
 SCHED_FEAT(SIS_UTIL, true)
+SCHED_FEAT(SIS_FAST, true)
 
 /*
  * Issue a WARN when we do multiple update_rq_clock() calls

--
Thanks and Regards,
Prateek

#74 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

Hello Chenyu,

Quoted 56 lines · Chen Yu
Quoted 18 lines

kernel/sched/fair.c static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool · line 7407

Quoted 13 lines

[..snip..]
/*
* Scan the LLC domain for idle CPUs; this is dynamically regulated by
* comparing the average scan cost (tracked in sd->avg_scan_cost) against the

kernel/sched/fair.c +2 -2
@@ -7384,10 +7402,9 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
 	if (sched_feat(SIS_UTIL)) {
 		sd_share = rcu_dereference(per_cpu(sd_llc_shared, target));
 		if (sd_share) {
-			/* because !--nr is the condition to stop scan */> -			nr = READ_ONCE(sd_share->nr_idle_scan) + 1;
+			nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan));
 			/* overloaded LLC is unlikely to have idle cpu/core */
-			if (nr == 1)
+			if (nr <= 0)

I was wondering if this would preserve the current behavior with SIS_FAST toggled off? Since the implementation below still does a "--nr <= 0" , wouldn't it effectively visit one CPU less overall now?

Quoted 7 lines

Have you tried something similar to the below hunk?

	/* because !--nr is the condition to stop scan */
	nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan)) + 1;
	if (nr == 1)
		return -1;

Yeah, right, to keep the scan depth consistent, the "+1" should be kept.

Quoted 6 lines

I agree with Mike that looking at slice to limit scan-depth seems odd. My experience with netperf is that the workload cares more about the server-client being co-located on the closest cache domain and by limiting scan-depth using slice, this is indirectly achieved since all the wakeups carry the WF_SYNc flag.

Exactly. This is the original motivation.

Quoted 4 lines

P.S. have you tried using the slice in __select_idle_cpu()? Similar to sched_idle_cpu() check, perhaps an additional sched_preempt_short_cpu() which compares rq->curr->se.slice with the waking task's slice and returs that cpu if SIS_SHORT can help run the workload quicker?

This is a good idea, it seems to be benefit PREEMPT_SHORT. If the customized task slice is introduced, we can leverage this hint for latency related optimization. Task wakeup is one thing, I can also think of other aspects, like idle load balance, etc. I'm not sure what is the proper usage of the task slice though, this is why I sent this RFC.

Quoted 7 lines

Note:
This will not work if the SIS scan itself is the largest overhead in the wakeup cycle and not the task placement itself. Previously during SIS_UTIL testing, to measure the overheads of scan vs placement, we would do a full scan but return the result that SIS_UTIL would have returned to determine the overhead of the search itself.

Quoted 3 lines

Regarding the task placement, do you mean the time between a task is enqueued and picked up? Do you have any recommendation which workload can expose the scan overhead most?

Sorry for not being clear here. From what I've observed in the past, there are two dimensions to slect_idle_sibling():

i) Placement: Final CPU select_idle_sibling() returns ii) Search: Do we find an idle core/CPU in select_idle_sibling()

In case of netperf, I've observed that i) is more important than ii) wherin a placement of client on same core/thread as that of the server results in better performance vs finding an idle CPU on a remote LLC. For hackbench/tbench, when runqueues are under high utilization (~75%), reduction in search time ii) seems to be more beneficial.

There was also a wakeup from IPI / without IPI angle that I never quite got to the bottom of that Mathieu has highlighted last year. I'll go get some more data on that front and give your patch a try. Expect results in a couple of days.

Thank you.

Quoted 8 lines

thanks,
Chenyu

Quoted 4 lines
Quoted 4 lines
kernel/sched/fair.c +0 -0
 				return -1;
 		}
 	}
[..snip..]

--
Thanks and Regards,
Prateek

#75 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

Quoted 63 lines · K Prateek Nayak

Hello Chenyu,

Quoted 56 lines · Chen Yu
Quoted 18 lines

kernel/sched/fair.c static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool · line 7407

Quoted 13 lines

[..snip..]
/*
* Scan the LLC domain for idle CPUs; this is dynamically regulated by
* comparing the average scan cost (tracked in sd->avg_scan_cost) against the

kernel/sched/fair.c +2 -2
@@ -7384,10 +7402,9 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
 	if (sched_feat(SIS_UTIL)) {
 		sd_share = rcu_dereference(per_cpu(sd_llc_shared, target));
 		if (sd_share) {
-			/* because !--nr is the condition to stop scan */> -			nr = READ_ONCE(sd_share->nr_idle_scan) + 1;
+			nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan));
 			/* overloaded LLC is unlikely to have idle cpu/core */
-			if (nr == 1)
+			if (nr <= 0)

I was wondering if this would preserve the current behavior with SIS_FAST toggled off? Since the implementation below still does a "--nr <= 0" , wouldn't it effectively visit one CPU less overall now?

Quoted 7 lines

Have you tried something similar to the below hunk?

	/* because !--nr is the condition to stop scan */
	nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan)) + 1;
	if (nr == 1)
		return -1;

Yeah, right, to keep the scan depth consistent, the "+1" should be kept.

Quoted 6 lines

I agree with Mike that looking at slice to limit scan-depth seems odd. My experience with netperf is that the workload cares more about the server-client being co-located on the closest cache domain and by limiting scan-depth using slice, this is indirectly achieved since all the wakeups carry the WF_SYNc flag.

Exactly. This is the original motivation.

Quoted 4 lines

P.S. have you tried using the slice in __select_idle_cpu()? Similar to sched_idle_cpu() check, perhaps an additional sched_preempt_short_cpu() which compares rq->curr->se.slice with the waking task's slice and returs that cpu if SIS_SHORT can help run the workload quicker?

This is a good idea, it seems to be benefit PREEMPT_SHORT. If the customized task slice is introduced, we can leverage this hint for latency related optimization. Task wakeup is one thing, I can also think of other aspects, like idle load balance, etc. I'm not sure what is the proper usage of the task slice though, this is why I sent this RFC.

Quoted 7 lines

Note:
This will not work if the SIS scan itself is the largest overhead in the wakeup cycle and not the task placement itself. Previously during SIS_UTIL testing, to measure the overheads of scan vs placement, we would do a full scan but return the result that SIS_UTIL would have returned to determine the overhead of the search itself.

Quoted 3 lines

Regarding the task placement, do you mean the time between a task is enqueued and picked up? Do you have any recommendation which workload can expose the scan overhead most?

Quoted 6 lines

Sorry for not being clear here. From what I've observed in the past, there are two dimensions to slect_idle_sibling():

i) Placement: Final CPU select_idle_sibling() returns ii) Search: Do we find an idle core/CPU in select_idle_sibling()

I see.

Quoted 3 lines

In case of netperf, I've observed that i) is more important than ii) wherin a placement of client on same core/thread as that of the server results in better performance vs finding an idle CPU on a remote LLC.

How about placement of client on same core/thread vs finding an idle CPU on a local LLC?

Quoted 3 lines

For hackbench/tbench, when runqueues are under high utilization (~75%), reduction in search time ii) seems to be more beneficial.

I can understand that hackbench is idle-cpu sensitive because it is MxN wakeup relationship and could result in task stacking easily. While for tbench, it should be similar to netperf, not sure why it does not fall into i) > ii) case. Is it because you were testing netperf TCP_RR(full-duplex) and tbench is half-duplex(the former has stronger cache locality requirement)?

Quoted 4 lines

There was also a wakeup from IPI / without IPI angle that I never quite got to the bottom of that Mathieu has highlighted last year. I'll go get some more data on that front and give your patch a try. Expect results in a couple of days.

Sure, I'd be glad to try your patch.

thanks,
Chenyu

#76 lore raw

Re: [RFC][PATCH 10/10] sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion

In reply to K Prateek Nayak
Quoted 66 lines · K Prateek Nayak

Hello Chenyu,

Quoted 56 lines · Chen Yu
Quoted 18 lines

Review static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool · line 7407

Quoted 13 lines

[..snip..]
/*
* Scan the LLC domain for idle CPUs; this is dynamically regulated by
* comparing the average scan cost (tracked in sd->avg_scan_cost) against the

(quoted diff) +2 -2
@@ -7384,10 +7402,9 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
 	if (sched_feat(SIS_UTIL)) {
 		sd_share = rcu_dereference(per_cpu(sd_llc_shared, target));
 		if (sd_share) {
-			/* because !--nr is the condition to stop scan */> -			nr = READ_ONCE(sd_share->nr_idle_scan) + 1;
+			nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan));
 			/* overloaded LLC is unlikely to have idle cpu/core */
-			if (nr == 1)
+			if (nr <= 0)

I was wondering if this would preserve the current behavior with SIS_FAST toggled off? Since the implementation below still does a "--nr <= 0" , wouldn't it effectively visit one CPU less overall now?

Quoted 7 lines

Have you tried something similar to the below hunk?

	/* because !--nr is the condition to stop scan */
	nr = adjust_idle_scan(p, READ_ONCE(sd_share->nr_idle_scan)) + 1;
	if (nr == 1)
		return -1;

Yeah, right, to keep the scan depth consistent, the "+1" should be kept.

Quoted 6 lines

I agree with Mike that looking at slice to limit scan-depth seems odd. My experience with netperf is that the workload cares more about the server-client being co-located on the closest cache domain and by limiting scan-depth using slice, this is indirectly achieved since all the wakeups carry the WF_SYNc flag.

Exactly. This is the original motivation.

Quoted 4 lines

P.S. have you tried using the slice in __select_idle_cpu()? Similar to sched_idle_cpu() check, perhaps an additional sched_preempt_short_cpu() which compares rq->curr->se.slice with the waking task's slice and returs that cpu if SIS_SHORT can help run the workload quicker?

This is a good idea, it seems to be benefit PREEMPT_SHORT. If the customized task slice is introduced, we can leverage this hint for latency related optimization. Task wakeup is one thing, I can also think of other aspects, like idle load balance, etc. I'm not sure what is the proper usage of the task slice though, this is why I sent this RFC.

Quoted 7 lines

Note:
This will not work if the SIS scan itself is the largest overhead in the wakeup cycle and not the task placement itself. Previously during SIS_UTIL testing, to measure the overheads of scan vs placement, we would do a full scan but return the result that SIS_UTIL would have returned to determine the overhead of the search itself.

Quoted 3 lines

Regarding the task placement, do you mean the time between a task is enqueued and picked up? Do you have any recommendation which workload can expose the scan overhead most?

Sorry for not being clear here. From what I've observed in the past, there are two dimensions to slect_idle_sibling():

Quoted 9 lines

i) Placement: Final CPU select_idle_sibling() returns ii) Search: Do we find an idle core/CPU in select_idle_sibling()

In case of netperf, I've observed that i) is more important than ii) wherin a placement of client on same core/thread as that of the server results in better performance vs finding an idle CPU on a remote LLC. For hackbench/tbench, when runqueues are under high utilization (~75%), reduction in search time ii) seems to be more beneficial.

Usually task stacking is not preferred because it hurts latency. But as we discovered on AMD and Intel's large servers, sometimes the overhead of task migration offset the benefit of running an idle CPU.

Inspired by your description on SIS_SHORT, and also Mike's feedback, I respin the SIS_SHORT patch, to consider the task's customized slice:

Leverage the WF_SYNC to achieve better cache locality.
If both the waker and the wakee are short duration tasks, wake up the wakee on waker's CPU directly. To avoid task stacking as much as possible:

1. Only when has_idle_core is false, SIS_SYNC takes effect.

2. Only if the user has shrinked the task slice, SIS_SYNC takes effect.

3. The waker must be the only runnable task on the runqueue.

4. The duration threshold is based on the CPU number within the LLC. The more CPUs there are, the lower the bar for the task to be treated as a short duration one, and vice versa.

threshold = sysctl_sched_migration_cost * llc_weight^2 / 256^2

threshold

   LLC_WEIGHT=8               0.5 usec
   LLC_WEIGHT=16              2 usec
   LLC_WEIGHT=32              8 usec
   LLC_WEIGHT=64              31 usec
   LLC_WEIGHT=128             125 usec
   LLC_WEIGHT=256             500 usec

5. Honor idle-CPU-first for CPUs share the L2 cache(Cluster). Only there is no idle CPU within the Cluster domain, SIS_SYNC takes effect.

Benchmark:
Tested on 4 platforms, significant throughput improvement on tbench, netperf, stress-ng, will-it-scale, and latency reduced of lmbench. No performance difference was observed in an Online Transaction Processing (OLTP) test.

Platform1, 240 CPUs, 2 sockets Intel(R) Xeon(R)
========================================================================
netperf
=======

case            	load    	baseline(std%)	compare%( std%)
TCP_RR          	60-threads	 1.00 (  1.11)	 +8.22 (  1.06)
TCP_RR          	120-threads	 1.00 (  1.74)	+38.71 (  0.74)
TCP_RR          	180-threads	 1.00 (  1.21)	+108.12 (  1.07)
TCP_RR          	240-threads	 1.00 (  4.75)	+160.98 (  4.01)
UDP_RR          	60-threads	 1.00 ( 11.70)	 +7.17 ( 10.89)
UDP_RR          	120-threads	 1.00 ( 10.82)	 +5.54 (  7.17)
UDP_RR          	180-threads	 1.00 (  9.77)	+10.29 ( 10.87)
UDP_RR          	240-threads	 1.00 ( 12.38)	 +4.45 ( 16.69)

tbench
======

case            	load    	baseline(std%)	compare%( std%)
loopback        	60-threads	 1.00 (  0.22)	 +8.67 (  0.12)
loopback        	120-threads	 1.00 (  0.20)	+42.12 (  1.16)
loopback        	180-threads	 1.00 (  0.16)	+96.36 (  1.34)
loopback        	240-threads	 1.00 (  0.23)	+122.68 (  4.87)

schbench
========
No noticeable difference of 99.0th wakeup/request latency, 50.0th RPS percentiles. schbench -m 2 -r 100

baseline sis_sync

Wakeup Latencies 99.0th usec 27 25

Request Latencies 99.0th usec        15376                         15376
RPS percentiles 50.0th               16608                         16608

Platform2, 48 CPUs 2 sockets Intel(R) Xeon(R) CPU E5-2697
========================================================================
lmbench3: lmbench3.PIPE.latency.us 33.8% improvement
lmbench3: lmbench3.AF_UNIX.sock.stream.latency.us 30.6% improvement

Platform3: 224 threads 2 sockets Intel(R) Xeon(R) Platinum 8480
=======================================================================
stress-ng: stress-ng.vm-rw.ops_per_sec 250.8% improvement
will-it-scale: will-it-scale.per_process_ops 42.1% improvement

Platform4: 288 CPUs, 2 sockets, 4 CPUs share the L2 cache, Intel(R) Xeon(R)
=========================================================================
netperf
=======

case            	load    	baseline(std%)	compare%( std%)
TCP_RR          	72-threads	 1.00 (  0.89)	 +6.14 (  0.95)
TCP_RR          	144-threads	 1.00 (  1.53)	 +5.18 (  1.24)
TCP_RR          	216-threads	 1.00 ( 10.76)	+23.31 (  1.67)
TCP_RR          	288-threads	 1.00 (  2.28)	 +1.72 (  2.10)
UDP_RR          	72-threads	 1.00 (  3.44)	 +0.42 (  3.11)
UDP_RR          	144-threads	 1.00 ( 14.53)	 +1.40 ( 16.11)
UDP_RR          	216-threads	 1.00 ( 17.30)	 +1.18 ( 23.00)
UDP_RR          	288-threads	 1.00 ( 21.30)	 -0.15 ( 20.14)
kernel/sched/fair.c +39 -5
index 1437c276c915..b2b838d499ea 100644
@@ -1003,7 +1003,7 @@ static void update_deadline(struct cfs_rq *cfs_rq, struct sched_entity *se)
 #include "pelt.h"
 #ifdef CONFIG_SMP
 
-static int select_idle_sibling(struct task_struct *p, int prev_cpu, int cpu);
+static int select_idle_sibling(struct task_struct *p, int prev_cpu, int cpu, int sync);
 static unsigned long task_h_load(struct task_struct *p);
 static unsigned long capacity_of(int cpu);
 
@@ -7410,16 +7410,36 @@ static inline int select_idle_smt(struct task_struct *p, struct sched_domain *sd
 
 #endif /* CONFIG_SCHED_SMT */
 
+static int short_task(struct task_struct *p, int llc)
+{
+	if (!p->se.custom_slice || p->se.slice >= sysctl_sched_base_slice)
+		return 0;
+
+	/*
+	 * Scale the threshold by the LLC CPU number.
+	 * The more CPUs there are, the more likely the
+	 * task is regarded as a small one.
+	 *
+	 */
+	if ((p->duration_avg << 16) >=
+	    (sysctl_sched_migration_cost * llc * llc))
+		return 0;
+
+	return 1;
+}
+
 /*
  * Scan the LLC domain for idle CPUs; this is dynamically regulated by
  * comparing the average scan cost (tracked in sd->avg_scan_cost) against the
  * average idle time for this rq (as found in rq->avg_idle).
  */
-static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool has_idle_core, int target)
+static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool has_idle_core, int target,
+			   int sync)
 {
 	struct cpumask *cpus = this_cpu_cpumask_var_ptr(select_rq_mask);
 	int i, cpu, idle_cpu = -1, nr = INT_MAX;
 	struct sched_domain_shared *sd_share;
+	int llc_weight = per_cpu(sd_llc_size, target);
 
 	cpumask_and(cpus, sched_domain_span(sd), p->cpus_ptr);
 
@@ -7458,6 +7478,20 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
 		}
 	}
 
+	/*
+	 * When reached here, next step is to scan idle CPUs that do not share L2.
+	 * However, the Core-to-Core cache latency could be large on big system.
+	 * Give target another chance if waker and wakee are mutually waking up
+	 * each other.
+	 */
+	if (sched_feat(SIS_SYNC) &&
+	    target == smp_processor_id() && !has_idle_core &&
+	    sync && this_rq()->nr_running <= 1 &&
+	    short_task(p, llc_weight) &&
+	    short_task(current, llc_weight)) {
+		return target;
+	}
+
 	for_each_cpu_wrap(cpu, cpus, target + 1) {
 		if (has_idle_core) {
 			i = select_idle_core(p, cpu, cpus, &idle_cpu);
@@ -7550,7 +7584,7 @@ static inline bool asym_fits_cpu(unsigned long util,
 /*
  * Try and locate an idle core/thread in the LLC cache domain.
  */
-static int select_idle_sibling(struct task_struct *p, int prev, int target)
+static int select_idle_sibling(struct task_struct *p, int prev, int target, int sync)
 {
 	bool has_idle_core = false;
 	struct sched_domain *sd;
@@ -7659,7 +7693,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 		}
 	}
 
-	i = select_idle_cpu(p, sd, has_idle_core, target);
+	i = select_idle_cpu(p, sd, has_idle_core, target, sync);
 	if ((unsigned)i < nr_cpumask_bits)
 		return i;
 
@@ -8259,7 +8293,7 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
 		new_cpu = sched_balance_find_dst_cpu(sd, p, cpu, prev_cpu, sd_flag);
 	} else if (wake_flags & WF_TTWU) { /* XXX always ? */
 		/* Fast path */
-		new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
+		new_cpu = select_idle_sibling(p, prev_cpu, new_cpu, sync);
 	}
 	rcu_read_unlock();
 
kernel/sched/features.h +1 -0
index 143f55df890b..7e5968d01dcb 100644
@@ -50,6 +50,7 @@ SCHED_FEAT(TTWU_QUEUE, true)
  * When doing wakeups, attempt to limit superfluous scans of the LLC domain.
  */
 SCHED_FEAT(SIS_UTIL, true)
+SCHED_FEAT(SIS_SYNC, true)
 
 /*
  * Issue a WARN when we do multiple update_rq_clock() calls
Signature · 5 lines
2.25.1



Preparation patch to record the task's duration:
include/linux/sched.h +3 -0
index 61591ac6eab6..9e8dfdc81048 100644
@@ -1339,6 +1339,9 @@ struct task_struct {
 	struct callback_head		cid_work;
 #endif
 
+	u64				prev_sleep_sum_runtime;
+	u64				duration_avg;
+
 	struct tlbflush_unmap_batch	tlb_ubc;
 
 	/* Cache last used pipe for splice(): */
kernel/sched/core.c +2 -0
index bcf2c4cc0522..c4288d613374 100644
@@ -4566,6 +4566,8 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p)
 	p->migration_pending = NULL;
 #endif
 	init_sched_mm_cid(p);
+	p->prev_sleep_sum_runtime = 0;
+	p->duration_avg = 0;
 }
 
 DEFINE_STATIC_KEY_FALSE(sched_numa_balancing);
kernel/sched/fair.c +12 -0
index 8a5b1ae0aa55..1437c276c915 100644
@@ -6833,6 +6833,15 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 
 static void set_next_buddy(struct sched_entity *se);
 
+static inline void dur_avg_update(struct task_struct *p)
+{
+	u64 dur;
+
+	dur = p->se.sum_exec_runtime - p->prev_sleep_sum_runtime;
+	p->prev_sleep_sum_runtime = p->se.sum_exec_runtime;
+	update_avg(&p->duration_avg, dur);
+}
+
 /*
  * The dequeue_task method is called before nr_running is
  * decreased. We remove the task from the rbtree and
@@ -6905,6 +6914,9 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 
 dequeue_throttle:
 	util_est_update(&rq->cfs, p, task_sleep);
+	if (task_sleep)
+		dur_avg_update(p);
+
 	hrtick_update(rq);
 }
 
Signature · 1 line
2.25.1
#77 lore raw
In reply to Peter Zijlstra

Hello Peter,

Quoted 31 lines · Peter Zijlstra

Hi all,

I'm slowly crawling back out of the hole and trying to get back to work. Availability is still low on my end, but I'll try and respond to some email.

Anyway, in order to get my feet wet again with sitting behind a computer, find here a few patches that should functionally complete the EEVDF journey.

This very much includes the new interface that exposes the extra parameter that EEVDF has. I've chosen to use sched_attr::sched_runtime for this over a nice-like value because some workloads actually know their slice length (can be dynamically measured in the same way as for deadline using CLOCK_THREAD_CPUTIME_ID) and using the real request size is much more effective than some relative measure.

[[ using too short a request size will increase job preemption overhead,
using too long a request size will decrease timeliness ]]

The whole delayed-dequeue thing is I think a fundamental thing that was missing from the EEVDF paper. Without something like this EEVDF will simply not work right. IIRC this was mentioned to me many years ago when people worked on BFQ iosched and ran into this same issue. Time had erased the critical aspect of this note and I had to re-discover it again.

Also, I think Ben expressed concern that preserving lag over long periods doesn't make sense a while back.

The implementation presented here is one that should work with our cgroup mess and keeps most of the ugly inside fair.c unlike previous versions which puked all over the core scheduler code.

Quoted 8 lines

Critically cfs-cgroup throttling is not tested, and cgroups are only tested in so far that a systemd infected machine now boots (took a bit).

Other than that, it works well enough to build the next kernel and it passes the few trivial latency-slice tests I ran.

Anyway, please have a poke and let me know...

Sorry for the delay, I was waiting for all the basic issues to get fixed before giving it a test but since OSPM is this week and folks would want to discuss the series, I'm sharing the results based on the current

git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git sched/eevdf

At the start of the testing the HEAD was at commit 3883d848e41a ("sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion") and I'm using commit 4c43611fe406 ("Merge branch 'tip/sched/urgent'") as the base for comparison.

tl;dr

Mixed bag of results. We have hackbench and netperf slightly unhappy. stream is as is, no surprises there, and tbench and schbench seems to have a swing from being unhappy to happy as we scale. I'll leave full results below including some experiments around sched_feat(RESPECT_SLICE) proposed here
https://lore.kernel.org/lkml/[email protected]/

o System Details

- 3rd Generation EPYC System
- 2 x 64C/128T
- NPS1 mode

o Kernels

incomplete: peterz/queue:sched/eevdf at commit 4c43611fe406
("Merge branch 'tip/sched/urgent'")

complete: peterz/queue:sched/eevdf at commit 3883d848e41a
("sched/eevdf: Use sched_attr::sched_runtime to set request/slice suggestion")

Note: The above mentioned tree is prove to force updates and the commit IDs reflect the state from when the branch was last updated on "2024-04-30"

o Results

==================================================================
Test : hackbench
Units : Normalized time in seconds
Interpretation: Lower is better
Statistic : AMean
==================================================================

Case:        incomplete[pct imp](CV)   complete[pct imp](CV)
 1-groups     1.00 [ -0.00]( 3.42)     1.32 [-32.19]( 2.87)
 2-groups     1.00 [ -0.00]( 1.28)     1.39 [-39.10]( 1.70)
 4-groups     1.00 [ -0.00]( 0.93)     1.50 [-49.85]( 1.28)
 8-groups     1.00 [ -0.00]( 1.00)     1.67 [-66.81]( 1.39)
16-groups     1.00 [ -0.00]( 2.69)     1.68 [-68.05]( 1.79)

==================================================================
Test : tbench
Units : Normalized throughput
Interpretation: Higher is better
Statistic : AMean
==================================================================

Clients:  incomplete[pct imp](CV)  complete[pct imp](CV)
    1     1.00 [  0.00]( 0.41)     0.98 [ -2.50]( 0.38)
    2     1.00 [  0.00]( 0.48)     0.96 [ -4.17]( 3.05)
    4     1.00 [  0.00]( 0.43)     0.96 [ -4.18]( 1.45)
    8     1.00 [  0.00]( 0.97)     0.93 [ -6.77]( 0.97)
   16     1.00 [  0.00]( 0.21)     0.91 [ -8.88]( 1.74)
   32     1.00 [  0.00]( 1.11)     0.92 [ -8.31]( 2.67)
   64     1.00 [  0.00]( 1.81)     0.96 [ -3.80]( 2.41)
  128     1.00 [  0.00]( 0.44)     1.02 [  2.43]( 0.92)
  256     1.00 [  0.00]( 0.26)     0.96 [ -4.35]( 1.07)
  512     1.00 [  0.00]( 0.12)     0.97 [ -2.87]( 0.64)
 1024     1.00 [  0.00]( 0.32)     0.97 [ -2.59]( 0.26)

==================================================================
Test : stream-10
Units : Normalized Bandwidth, MB/s
Interpretation: Higher is better
Statistic : HMean
==================================================================

Test:    incomplete[pct imp](CV)   complete[pct imp](CV)
 Copy     1.00 [  0.00](11.28)     0.80 [-20.19](15.70)
Scale     1.00 [  0.00]( 5.61)     0.99 [ -1.27]( 6.81)
  Add     1.00 [  0.00]( 6.76)     0.91 [ -8.97]( 7.97)
Triad     1.00 [  0.00]( 4.76)     0.88 [-12.27]( 8.45)

==================================================================
Test : stream-100
Units : Normalized Bandwidth, MB/s
Interpretation: Higher is better
Statistic : HMean
==================================================================

Test:    incomplete[pct imp](CV)   complete[pct imp](CV)
 Copy     1.00 [  0.00]( 3.28)     0.95 [ -4.61]( 3.81)
Scale     1.00 [  0.00]( 2.00)     0.99 [ -0.87]( 5.04)
  Add     1.00 [  0.00]( 1.44)     0.99 [ -0.51]( 2.12)
Triad     1.00 [  0.00]( 2.41)     1.01 [  0.63]( 1.45)

==================================================================
Test : netperf
Units : Normalized Througput
Interpretation: Higher is better
Statistic : AMean
==================================================================

Clients:     incomplete[pct imp](CV)    complete[pct imp](CV)
 1-clients     1.00 [  0.00]( 0.33)     0.99 [ -0.85]( 1.65)
 2-clients     1.00 [  0.00]( 0.58)     0.99 [ -1.18]( 0.35)
 4-clients     1.00 [  0.00]( 0.32)     0.99 [ -0.78]( 1.27)
 8-clients     1.00 [  0.00]( 0.31)     0.99 [ -1.31]( 0.97)
16-clients     1.00 [  0.00]( 0.53)     0.98 [ -1.95]( 1.11)
32-clients     1.00 [  0.00]( 0.50)     0.97 [ -2.94]( 1.29)
64-clients     1.00 [  0.00]( 1.71)     0.95 [ -4.86]( 1.14)
128-clients    1.00 [  0.00]( 0.82)     0.99 [ -0.67]( 2.55)
256-clients    1.00 [  0.00]( 3.88)     0.92 [ -7.65]( 3.27)
512-clients    1.00 [  0.00](43.49)     0.94 [ -6.35](49.81)

==================================================================
Test : schbench (Old)
Units : Normalized 99th percentile latency in us
Interpretation: Lower is better
Statistic : Median
==================================================================

#workers: incomplete[pct imp](CV)   complete[pct imp](CV)
  1        1.00 [ -0.00](34.42)     0.76 [ 23.68](31.33)
  2        1.00 [ -0.00]( 6.45)     0.83 [ 16.67]( 5.84)
  4        1.00 [ -0.00](14.87)     0.77 [ 22.73](16.44)
  8        1.00 [ -0.00](11.30)     1.07 [ -6.52]( 9.55)
 16        1.00 [ -0.00]( 6.33)     1.03 [ -3.45]( 6.11)
 32        1.00 [ -0.00]( 2.18)     0.94 [  6.25]( 3.77)
 64        1.00 [ -0.00]( 7.23)     0.99 [  0.51]( 5.93)
128        1.00 [ -0.00]( 6.76)     0.95 [  5.00]( 5.17)
256        1.00 [ -0.00](11.05)     1.22 [-22.33]( 9.70)
512        1.00 [ -0.00]( 2.15)     0.77 [ 23.20]( 2.81)

--

o Regression Breakdown

Following are some data I could gather for the regression observed:

- Hackbench

The runtime of hackbench with EEVDF Complete is 24.81% more that that on the current base kernel. Following is the IBS profile comparing EEVDF Complete on the left to current base on the right:

Command: sudo perf record -a -e ibs_op// -- perf bench sched messaging -p -t -l 100000 -g 1

Overhead  Command          Shared Object             Symbol                                             Overhead  Command          Shared Object             Symbol
   4.42%  sched-messaging  [kernel.kallsyms]         [k] osq_lock                                          6.30%  sched-messaging  [kernel.kallsyms]         [k] osq_lock
   3.76%  sched-messaging  [kernel.kallsyms]         [k] _copy_from_iter                                   4.06%  sched-messaging  [kernel.kallsyms]         [k] _copy_to_iter
   3.67%  sched-messaging  [kernel.kallsyms]         [k] srso_alias_safe_ret                               3.98%  sched-messaging  [kernel.kallsyms]         [k] _copy_from_iter
   3.18%  sched-messaging  [kernel.kallsyms]         [k] _copy_to_iter                                     3.85%  sched-messaging  [kernel.kallsyms]         [k] mutex_spin_on_owner
   2.80%  sched-messaging  [kernel.kallsyms]         [k] pipe_read                                         3.50%  sched-messaging  [kernel.kallsyms]         [k] srso_alias_safe_ret
   2.72%  sched-messaging  [kernel.kallsyms]         [k] mutex_spin_on_owner                               3.33%  sched-messaging  [kernel.kallsyms]         [k] inode_needs_update_time
   2.69%  sched-messaging  [kernel.kallsyms]         [k] pipe_write                                        2.39%  sched-messaging  [kernel.kallsyms]         [k] pipe_read
   2.49%  sched-messaging  [kernel.kallsyms]         [k] syscall_exit_to_user_mode                         2.36%  sched-messaging  [kernel.kallsyms]         [k] syscall_exit_to_user_mode
   2.40%  swapper          [kernel.kallsyms]         [k] poll_idle                     <----               2.30%  sched-messaging  [kernel.kallsyms]         [k] pipe_write
   2.31%  sched-messaging  [kernel.kallsyms]         [k] inode_needs_update_time                           2.23%  sched-messaging  [kernel.kallsyms]         [k] mutex_lock
   2.14%  sched-messaging  [kernel.kallsyms]         [k] srso_alias_return_thunk                           2.07%  sched-messaging  [kernel.kallsyms]         [k] update_sd_lb_stats.constprop.0
   2.13%  sched-messaging  [kernel.kallsyms]         [k] aa_file_perm                                      2.02%  sched-messaging  [kernel.kallsyms]         [k] ksys_write
   1.99%  sched-messaging  [kernel.kallsyms]         [k] native_queued_spin_lock_slowpath                  2.00%  sched-messaging  [kernel.kallsyms]         [k] aa_file_perm
   1.99%  sched-messaging  [kernel.kallsyms]         [k] do_syscall_64                                     1.98%  sched-messaging  libc.so.6                 [.] __GI___libc_write
   1.92%  sched-messaging  [kernel.kallsyms]         [k] ksys_write                                        1.94%  sched-messaging  [kernel.kallsyms]         [k] do_syscall_64
   1.90%  sched-messaging  [kernel.kallsyms]         [k] __fdget_pos                                       1.92%  sched-messaging  [kernel.kallsyms]         [k] srso_alias_return_thunk
   1.89%  sched-messaging  [kernel.kallsyms]         [k] mutex_lock                                        1.76%  sched-messaging  [kernel.kallsyms]         [k] entry_SYSCALL_64
   1.88%  sched-messaging  [kernel.kallsyms]         [k] entry_SYSCALL_64                                  1.62%  sched-messaging  [kernel.kallsyms]         [k] __mutex_lock.constprop.0
   1.62%  sched-messaging  libc.so.6                 [.] __GI___libc_write                                 1.48%  sched-messaging  [kernel.kallsyms]         [k] __fdget_pos
   1.61%  sched-messaging  [kernel.kallsyms]         [k] vfs_read                                          1.47%  sched-messaging  [kernel.kallsyms]         [k] vfs_read
   1.54%  sched-messaging  [kernel.kallsyms]         [k] apparmor_file_permission                          1.31%  sched-messaging  [kernel.kallsyms]         [k] native_queued_spin_lock_slowpath
   1.43%  sched-messaging  [kernel.kallsyms]         [k] rep_movs_alternative                              1.29%  sched-messaging  [kernel.kallsyms]         [k] apparmor_file_permission
   1.40%  sched-messaging  [kernel.kallsyms]         [k] entry_SYSCALL_64_after_hwframe                    1.25%  sched-messaging  libc.so.6                 [.] read
   1.36%  sched-messaging  [kernel.kallsyms]         [k] ksys_read                                         1.22%  sched-messaging  [kernel.kallsyms]         [k] mutex_unlock
   1.35%  sched-messaging  [kernel.kallsyms]         [k] vfs_write                                         1.18%  sched-messaging  [kernel.kallsyms]         [k] rep_movs_alternative
   1.24%  sched-messaging  libc.so.6                 [.] read                                              1.17%  sched-messaging  [kernel.kallsyms]         [k] entry_SYSCALL_64_after_hwframe
   1.23%  sched-messaging  [kernel.kallsyms]         [k] srso_alias_untrain_ret                            1.11%  sched-messaging  [kernel.kallsyms]         [k] atime_needs_update
   1.15%  sched-messaging  [kernel.kallsyms]         [k] __mutex_lock.constprop.0                          1.10%  sched-messaging  [kernel.kallsyms]         [k] vfs_write
   1.15%  sched-messaging  [kernel.kallsyms]         [k] atime_needs_update                                1.08%  sched-messaging  [kernel.kallsyms]         [k] ksys_read
   1.11%  swapper          [kernel.kallsyms]         [k] psi_group_change                                  1.03%  sched-messaging  [kernel.kallsyms]         [k] srso_alias_untrain_ret
   1.03%  sched-messaging  [kernel.kallsyms]         [k] copy_page_from_iter                               1.01%  swapper          [kernel.kallsyms]         [k] psi_group_change
   0.98%  sched-messaging  [kernel.kallsyms]         [k] update_sd_lb_stats.constprop.0                    0.96%  sched-messaging  [kernel.kallsyms]         [k] copy_page_from_iter
   0.95%  sched-messaging  [kernel.kallsyms]         [k] psi_group_change                                  0.93%  sched-messaging  [kernel.kallsyms]         [k] psi_group_change
   0.91%  sched-messaging  [kernel.kallsyms]         [k] copy_page_to_iter                                 0.79%  sched-messaging  [kernel.kallsyms]         [k] copy_page_to_iter
   0.89%  sched-messaging  [kernel.kallsyms]         [k] rw_verify_area                                    0.76%  swapper          [kernel.kallsyms]         [k] poll_idle		<------ poll idle jumps up in case of
   0.86%  swapper          [kernel.kallsyms]         [k] acpi_processor_ffh_cstate_enter                   0.71%  sched-messaging  [kernel.kallsyms]         [k] rw_verify_area			EEVDF complete
   0.77%  sched-messaging  [kernel.kallsyms]         [k] mutex_unlock                                      0.71%  swapper          [kernel.kallsyms]         [k] acpi_processor_ffh_cstate_enter
   0.77%  sched-messaging  [kernel.kallsyms]         [k] security_file_permission                          0.69%  sched-messaging  [kernel.kallsyms]         [k] try_to_wake_up
   0.73%  sched-messaging  [kernel.kallsyms]         [k] select_task_rq_fair                               0.67%  sched-messaging  [kernel.kallsyms]         [k] select_task_rq_fair
   0.71%  sched-messaging  [kernel.kallsyms]         [k] __schedule                                        0.63%  sched-messaging  [kernel.kallsyms]         [k] __schedule
   0.70%  sched-messaging  [kernel.kallsyms]         [k] entry_SYSRETQ_unsafe_stack                        0.61%  sched-messaging  [kernel.kallsyms]         [k] entry_SYSRETQ_unsafe_stack
   0.59%  sched-messaging  [kernel.kallsyms]         [k] syscall_return_via_sysret                         0.61%  sched-messaging  [kernel.kallsyms]         [k] security_file_permission
   0.57%  sched-messaging  [kernel.kallsyms]         [k] dequeue_entity                                    0.52%  sched-messaging  [kernel.kallsyms]         [k] update_load_avg
   0.54%  sched-messaging  [kernel.kallsyms]         [k] current_time                                      0.51%  sched-messaging  [kernel.kallsyms]         [k] native_sched_clock

--

Further pinning the workload to one LLC domain (CCX) shows poll_idle jump to top:

Command: sudo perf record -C 0-7,128-131 -e ibs_op// -- taskset -c 0-7,128-131 perf bench sched messaging -p -t -l 100000 -g 1

Overhead  Command          Shared Object         Symbol                                                 Overhead  Command          Shared Object          Symbol
   9.14%  swapper          [kernel.kallsyms]     [k] poll_idle				<-----             3.55%  sched-messaging  [kernel.kallsyms]      [k] pipe_read
   3.02%  sched-messaging  [kernel.kallsyms]     [k] native_queued_spin_lock_slowpath                      3.53%  sched-messaging  [kernel.kallsyms]      [k] pipe_write
   3.00%  sched-messaging  [kernel.kallsyms]     [k] pipe_read                                             3.48%  sched-messaging  [kernel.kallsyms]      [k] srso_alias_safe_ret
   2.97%  sched-messaging  [kernel.kallsyms]     [k] pipe_write                                            3.37%  sched-messaging  [kernel.kallsyms]      [k] _copy_from_iter
   2.95%  sched-messaging  [kernel.kallsyms]     [k] srso_alias_safe_ret                                   3.15%  sched-messaging  [kernel.kallsyms]      [k] syscall_exit_to_user_mode
   2.83%  sched-messaging  [kernel.kallsyms]     [k] _copy_from_iter                                       2.76%  sched-messaging  [kernel.kallsyms]      [k] native_queued_spin_lock_slowpath
   2.56%  sched-messaging  [kernel.kallsyms]     [k] syscall_exit_to_user_mode                             2.62%  sched-messaging  [kernel.kallsyms]      [k] srso_alias_return_thunk
   2.23%  sched-messaging  [kernel.kallsyms]     [k] srso_alias_return_thunk                               2.59%  sched-messaging  [kernel.kallsyms]      [k] do_syscall_64
   2.21%  sched-messaging  [kernel.kallsyms]     [k] do_syscall_64                                         2.44%  sched-messaging  [kernel.kallsyms]      [k] __fdget_pos
   2.05%  sched-messaging  [kernel.kallsyms]     [k] entry_SYSCALL_64                                      2.34%  sched-messaging  [kernel.kallsyms]      [k] entry_SYSCALL_64
   2.04%  sched-messaging  [kernel.kallsyms]     [k] __fdget_pos                                           2.20%  sched-messaging  [kernel.kallsyms]      [k] _copy_to_iter
   1.87%  sched-messaging  [kernel.kallsyms]     [k] _copy_to_iter                                         2.18%  sched-messaging  [kernel.kallsyms]      [k] aa_file_perm
   1.82%  sched-messaging  [kernel.kallsyms]     [k] aa_file_perm                                          2.16%  sched-messaging  [kernel.kallsyms]      [k] vfs_read
   1.67%  sched-messaging  [kernel.kallsyms]     [k] apparmor_file_permission                              2.08%  swapper          [kernel.kallsyms]      [k] poll_idle   			<-----
   1.65%  sched-messaging  [kernel.kallsyms]     [k] vfs_read                                              2.03%  sched-messaging  [kernel.kallsyms]      [k] apparmor_file_permission
   1.63%  sched-messaging  [kernel.kallsyms]     [k] entry_SYSCALL_64_after_hwframe                        2.00%  sched-messaging  [kernel.kallsyms]      [k] entry_SYSCALL_64_after_hwframe
   1.59%  swapper          [kernel.kallsyms]     [k] psi_group_change                                      1.85%  sched-messaging  [kernel.kallsyms]      [k] rep_movs_alternative
   1.56%  sched-messaging  [kernel.kallsyms]     [k] ksys_write                                            1.83%  sched-messaging  [kernel.kallsyms]      [k] vfs_write
   1.54%  sched-messaging  [kernel.kallsyms]     [k] ksys_read                                             1.82%  sched-messaging  [kernel.kallsyms]      [k] ksys_write
   1.51%  sched-messaging  [kernel.kallsyms]     [k] vfs_write                                             1.72%  sched-messaging  [kernel.kallsyms]      [k] srso_alias_untrain_ret
   1.49%  sched-messaging  [kernel.kallsyms]     [k] rep_movs_alternative                                  1.71%  sched-messaging  libc.so.6              [.] __GI___libc_write
   1.46%  sched-messaging  libc.so.6             [.] __GI___libc_write                                     1.70%  sched-messaging  [kernel.kallsyms]      [k] ksys_read
   1.42%  sched-messaging  [kernel.kallsyms]     [k] srso_alias_untrain_ret                                1.58%  sched-messaging  [kernel.kallsyms]      [k] atime_needs_update
   1.30%  sched-messaging  libc.so.6             [.] read                                                  1.56%  sched-messaging  libc.so.6              [.] read
   1.24%  sched-messaging  [kernel.kallsyms]     [k] atime_needs_update                                    1.56%  sched-messaging  [kernel.kallsyms]      [k] mutex_lock
   1.20%  sched-messaging  [kernel.kallsyms]     [k] mutex_lock                                            1.35%  sched-messaging  [kernel.kallsyms]      [k] psi_group_change
   1.04%  sched-messaging  [kernel.kallsyms]     [k] psi_group_change                                      1.26%  sched-messaging  [kernel.kallsyms]      [k] copy_page_to_iter
   1.03%  sched-messaging  [kernel.kallsyms]     [k] inode_needs_update_time                               1.20%  sched-messaging  [kernel.kallsyms]      [k] select_task_rq_fair
   0.99%  sched-messaging  [kernel.kallsyms]     [k] rw_verify_area                                        1.16%  sched-messaging  [kernel.kallsyms]      [k] inode_needs_update_time
   0.98%  sched-messaging  [kernel.kallsyms]     [k] copy_page_to_iter                                     1.16%  sched-messaging  [kernel.kallsyms]      [k] rw_verify_area
   0.95%  sched-messaging  [kernel.kallsyms]     [k] select_task_rq_fair                                   1.12%  sched-messaging  [kernel.kallsyms]      [k] copy_page_from_iter
   0.95%  sched-messaging  [kernel.kallsyms]     [k] copy_page_from_iter                                   1.06%  swapper          [kernel.kallsyms]      [k] psi_group_change
   0.92%  sched-messaging  [kernel.kallsyms]     [k] __schedule                                            1.04%  sched-messaging  [kernel.kallsyms]      [k] security_file_permission
   0.85%  sched-messaging  [kernel.kallsyms]     [k] security_file_permission                              0.96%  sched-messaging  [kernel.kallsyms]      [k] try_to_wake_up
   0.68%  swapper          [kernel.kallsyms]     [k] do_idle                                               0.86%  sched-messaging  [kernel.kallsyms]      [k] entry_SYSRETQ_unsafe_stack
   0.67%  sched-messaging  [kernel.kallsyms]     [k] dequeue_entity                                        0.80%  sched-messaging  [kernel.kallsyms]      [k] mutex_spin_on_owner
   0.67%  sched-messaging  [kernel.kallsyms]     [k] entry_SYSRETQ_unsafe_stack                            0.79%  sched-messaging  [kernel.kallsyms]      [k] __schedule
   0.66%  sched-messaging  [kernel.kallsyms]     [k] update_load_avg                                       0.79%  sched-messaging  [kernel.kallsyms]      [k] syscall_return_via_sysret
   0.65%  sched-messaging  [kernel.kallsyms]     [k] syscall_return_via_sysret                             0.75%  sched-messaging  [kernel.kallsyms]      [k] current_time
   0.62%  sched-messaging  [kernel.kallsyms]     [k] mutex_spin_on_owner                                   0.66%  sched-messaging  [kernel.kallsyms]      [k] __mutex_lock.constprop.0
   0.60%  swapper          [kernel.kallsyms]     [k] acpi_processor_ffh_cstate_enter                       0.63%  sched-messaging  perf                   [.] sender
   0.60%  swapper          [kernel.kallsyms]     [k] __schedule                                            0.63%  sched-messaging  [kernel.kallsyms]      [k] update_load_avg
   0.59%  sched-messaging  [kernel.kallsyms]     [k] current_time                                          0.59%  sched-messaging  perf                   [.] receiver
   0.54%  swapper          [kernel.kallsyms]     [k] enqueue_entity                                        0.56%  sched-messaging  [kernel.kallsyms]      [k] file_update_time
   0.52%  swapper          [kernel.kallsyms]     [k] menu_select                                           0.55%  sched-messaging  [kernel.kallsyms]      [k] touch_atime
   0.50%  sched-messaging  perf                  [.] sender                                                0.52%  sched-messaging  [kernel.kallsyms]      [k] x64_sys_call

--

This jump up of poll_idle seems to be a higher order behavior change as a result of Complete EEVDF. Looking at "sched/fair: Implement delayed dequeue", can there be a case where ttwu_runnable() in try_to_wake_up() can stop task migration on wakeup and queue it on a busy queue where it was marked for a dealyed dequeue?

When looking at per-task statistics from sched-scoreboard, I see a 41% decrease in nr_migrations going from base to Complete EEVDF. I also see the cumulative wait sum shoot up by 88%.

comm: sched-messaging incomplete complete %diff
Average of avg_atom 0.118341601 0.036585796 -69%
Average of avg_per_cpu 0.446550378 0.668355323 50%
Sum of nr_migrations 1414593 829870 -41% *
Sum of nr_switches 4013420 3563675 -11%
Sum of nr_voluntary_switches 3912865 3562547 -9%
Sum of nr_involuntary_switches 100555 1128 -99%
Sum of nr_wakeups 3913621 3567097 -9%

Sum of sum_exec_runtime	                95557.37505	101666.7651	  6%
Sum of sum_idle_runtime	                1928546.553	975487.7551	-49%
Sum of wait_sum	                        69884.24284	131511.9207	 88%	*

For tbench and netperf I could not get any concrete culprit from perf with IBS or per-task stats. I'll continue digging. However, it could just be similar higher order effect where poll_idle steals cycles from smt sibling running the task.

---------------------------------------------------------
- Experiments with sched_attr::sched_runtime on SPECjbb -
---------------------------------------------------------

I tried some experiments with sched_attr:sched_runtime on SPECjbb. With the base kernel, I was not able see any changes in the SPECjbb results with various sched_attr::sched_runtime set for various threads. However, with Complete EEVDF (contains RESPECT_SLICE), I'm able to eek out some performance improvements on the workload.

Background information on the threads of SPECjbb - txinject drives the workload by injecting transaction. MapReduce, ForkJoinPool, and the Backend threads are responsible for the main business logic of transaction processing. There are GC threads from Java and a compiler thread for what I presume is some form of JIT compilation.

I've used following naive strategy:

- Set a very small slice to txinject since I would like to push as much load as possible, as quickly as possible.

- Very small slice to ForkJoinPool and MapReduce threads so they can go ahead and pass the load to backend threads.

- Very small slice to compiler threads so thay can unblock whatever is waiting on the JIT compilation.

- A very large slice to the backend thread so it can process as much data as possible in one go.

- Maximum slice size to the GC thread purely because it should not interrupt other processes.

With the above setting, I see the following trend:
EEVDF Complete vs base
Max-jOPS +7.05%
Critical-jOPS -15.62%

The key takeaway of the experiment is that, with EEVDF complete, I'm actually starting to see some behavior difference with different values unlike with the current tip:Sched/core. I believe it is RESPECT_SLICE helping to keep tasks with large slices on the runqueue beyond the 0-lag point.

Note: These results are from naive experiments. I'm sure looking at the per-task statistics and deciding on the slices more judiciously can further benefit the workload.

On a parting note, it was not very straightforward to set these values outside of the application without any application modification. In my case, I was consuming "task_rename" evvents via trace pipe to find the relevant pids to then tag.

------------------------------------
- DeathStarBench and RESPECT_SLICE -
------------------------------------

Following is the trend I see for RESPECT_SLICE on DeathStarBench:

Pinning #instances EEVDF_COMPLETE
1LLC 1 9%
2LLC 2 3%
4LLC 3 -1%
8LLC 6 -5%

Dropping RESPECT_SLICE and using my original RUN_TO_PARITY_WAKEUP
https://lore.kernel.org/lkml/[email protected]/
still shows regression for larger instances pinned to larger number of LLC domains with EEVDF Complete. This remains to be investigated as well.

I'll update the thread with more information in the coming days. --
Thanks and Regards,
Prateek