Core scheduling v6
Subsystems (3)
Re: [RFC PATCH 14/16] irq: Add support for core-wide protection of IRQ and softirq
In reply to Li, AubreyQuoted 12 lines · Aubrey
On 2020/7/1 5:32, Vineeth Remanan Pillai wrote:
We saw a lot of soft lockups on the screen when we tested v6.
[ 186.527883] watchdog: BUG: soft lockup - CPU#86 stuck for 22s! [uperf:5551] [ 186.535884] watchdog: BUG: soft lockup - CPU#87 stuck for 22s! [uperf:5444] [ 186.555883] watchdog: BUG: soft lockup - CPU#89 stuck for 22s! [uperf:5547] [ 187.547884] rcu: INFO: rcu_sched self-detected stall on CPU [ 187.553760] rcu: 40-....: (14997 ticks this GP) idle=49a/1/0x4000000000000002 softirq=1711/1711 fqs=7279 [ 187.564685] NMI watchdog: Watchdog detected hard LOCKUP on cpu 14 [ 187.564723] NMI watchdog: Watchdog detected hard LOCKUP on cpu 38
Quoted 3 lines
The problem is gone when we reverted this patch. We are running multiple uperf threads(equal to cpu number) in a cgroup with coresched enabled. This is 100% reproducible on our side.
ROTFL. I just predicted that from staring at the patch ....
Re: [RFC PATCH 14/16] irq: Add support for core-wide protection of IRQ and softirq
Quoted 13 lines · Thomas Gleixner
Quoted 12 lines · Aubrey
On 2020/7/1 5:32, Vineeth Remanan Pillai wrote:
We saw a lot of soft lockups on the screen when we tested v6.
[ 186.527883] watchdog: BUG: soft lockup - CPU#86 stuck for 22s! [uperf:5551] [ 186.535884] watchdog: BUG: soft lockup - CPU#87 stuck for 22s! [uperf:5444] [ 186.555883] watchdog: BUG: soft lockup - CPU#89 stuck for 22s! [uperf:5547] [ 187.547884] rcu: INFO: rcu_sched self-detected stall on CPU [ 187.553760] rcu: 40-....: (14997 ticks this GP) idle=49a/1/0x4000000000000002 softirq=1711/1711 fqs=7279 [ 187.564685] NMI watchdog: Watchdog detected hard LOCKUP on cpu 14 [ 187.564723] NMI watchdog: Watchdog detected hard LOCKUP on cpu 38
Quoted 5 lines
Quoted 3 lines
The problem is gone when we reverted this patch. We are running multiple uperf threads(equal to cpu number) in a cgroup with coresched enabled. This is 100% reproducible on our side.
ROTFL. I just predicted that from staring at the patch ....
Yes, sorry. It got fixed in the below patch which I was about to share before you sent this email. However, it still has (not yet) addressed the new comments from you we have to discuss (will reply to that email).
Its purely a dirty test patch but it makes Aubrey's deadlock go away. Basically, I moved the waiting code to prepare_exit_to_usermode() which removes the lock vs wait dependencies. I will work on moving the code to the right place based on the suggestions in the other email.
thanks,
- Joel
Re: [RFC PATCH 14/16] irq: Add support for core-wide protection of IRQ and softirq
In reply to Vineeth Remanan PillaiVineeth, Joel!
Quoted 7 lines · Vineeth Remanan Pillai
From: "Joel Fernandes (Google)" <[email protected]>
Lastly, we also check in the schedule loop if we are about to schedule an untrusted process while the core is in such a state. This is possible if a trusted thread enters the scheduler by way of yielding CPU. This would involve no transitions through the
irq_exit()point to do any waiting, so we have to explicitly do the waiting there.
Lot's of 'we' and 'this patch' in this changelog. Care to read Documentation/process/submitting-patches.rst ?
Quoted 11 lines
Every attempt is made to prevent a busy-wait unnecessarily, and in testing on real-world ChromeOS usecases, it has not shown a performance drop. In ChromeOS, with this and the rest of the core scheduling patchset, we see around a 300% improvement in key press latencies into Google docs when Camera streaming is running simulatenously (90th percentile latency of ~150ms drops to ~50ms).
This fetaure is controlled by the build time config option
CONFIG_SCHED_CORE_IRQ_PAUSEand is enabled by default. There is also a kernel boot parameter 'sched_core_irq_pause' to enable/disable the feature at boot time. Default is enabled at boot time.
Enabled by default? At least that wants to depend on the enablement of core scheduling at boot time.
kernel/softirq.c static inline bool lockdep_softirq_start(void) { return false; } · line 251
Quoted 4 lines
kernel/softirq.c +3
-0+#ifdef CONFIG_SCHED_CORE_IRQ_PAUSE +DEFINE_STATIC_KEY_TRUE (sched_core_irq_pause); +static int __init set_sched_core_irq_pause (char *str)
Gah. newlines are overrated, right? Glueing variable declarations and function declarations next to each other is unreadable.
kernel/softirq.c static inline bool lockdep_softirq_start(void) { return false; } · line 254
Quoted 3 lines
kernel/softirq.c +3
-0+{ +unsigned long val =0 ; +if (!str)
Missing newline between declaration and code. checkpatch.pl has told you so.
kernel/softirq.c static inline bool lockdep_softirq_start(void) { return false; } · line 257
Quoted 3 lines
kernel/softirq.c +3
-0+return 0 ; + + val =simple_strtoul (str, &str,0 );
Huch? Why are you using simple_strtoul() for parsing a boolean and what's the point of setting the **end argument to &str?
Review
Quoted 2 lines
(quoted diff) +2
-0+ + if (val == 0)
if (!val)
is the usual kernel coding style.
Quoted 12 lines
kernel/softirq.c +7
-0+static_branch_disable (&sched_core_irq_pause); + +return 1 ; +} +__setup ("sched_core_irq_pause=" , set_sched_core_irq_pause); +#endif + asmlinkage __visible void __softirq_entry __do_softirq(void) {unsigned long end = jiffies +MAX_SOFTIRQ_TIME ;@@ -273,6 +291,16 @@ asmlinkage __visible void __softirq_entry __do_softirq(void)/* Reset the pending bitmask before enabling irqs */
kernel/softirq.c asmlinkage __visible void __softirq_entry __do_softirq(void) · line 301
Quoted 10 lines
kernel/softirq.c +8
-0set_softirq_pending (0 ); +#ifdef CONFIG_SCHED_CORE_IRQ_PAUSE +/* +* Core scheduling mitigations require entry into softirq to send stall +* IPIs to sibling hyperthreads if needed (ex, sibling is running +* untrusted task). If we are here from irq_exit(), no IPIs are sent. +*/ +if (static_branch_likely (&sched_core_irq_pause)) +sched_core_irq_enter ();
So you already have a #ifdef CONFIG_SCHED_CORE_IRQ_PAUSE section above. Why on earth can't you just provide helper functions and stubs for the CONFIG_SCHED_CORE_IRQ_PAUSE=n case there instead of sprinkling #ifdeffery all over the place?
kernel/softirq.c asmlinkage __visible void __softirq_entry __do_softirq(void) · line 304
Quoted 3 lines
kernel/softirq.c +2
-0+#endif +local_irq_enable ();
How is __do_softirq() relevant at all?
__do_softirq() is called from:
1) the tail of interrupt handling
2) the tail of local_bh_enable()
3) the ksoftirq thread
None of this is relevant for what you are trying to do:
#1: Iscovered by the interrupt handling code simply because this nests
into the interrupt handling.
#2: local_bh_enable() only ends up there when invoked from thread
context.
This would only make sense if one sibling is in the kernel and the other in user space or guest mode. Isn't that forbidden by core scheduling in the first place?
#3: See #2
But what's worse is that this is called from an interrupt disabled region. So you brilliantly created a trivial source of livelocks. stomp_machine() being the most obvious candidate. But it's easy enough to come up with more subtle ones.
The point is that you want to bring out the sibling thread from user space or guest mode when the other HT thread:
A) invokes a syscall or traps into the hypervisor which is semantically equivalent (not code wise).
B) handles an interrupt/exception
So #A covers #2 and #3 above and #B covers #1 above.
But you have the same livelock problem with doing this core wait thingy within any interrupt disabled region.
What you really want to do is on syscall and interrupt/exception entry:
if (other_sibling_in_user_or_guest()) send_IPI();
Of course it's conveniant to piggy pack that onto the reschedule IPI, but that sucks. What you want is a dedicated IPI which does:
magic_ipi()
set_thread_flag(TIF_CORE_SCHED_EXIT);
And on the exit side you want to look at the entry code generalisation I just posted:
https://lore.kernel.org/r/[email protected]
Add TIF_CORE_SCHED_EXIT to EXIT_TO_USER_MODE_WORK and to EXIT_TO_GUEST_MODE_WORK and handle it in exit_to_guest_mode_work() and exit_to_user_mode_loop() with interrupts enabled.
Trying to do this in fully interrupt disabled context is just a recipe for disaster.
The entry case condition wants to have a TIF bit as well, i.e.
if (thread_test_bit(TIF_CORE_SCHED_REQUIRED) {
sched_ipi_dance() {
if (other_sibling_in_user_or_guest())
send_IPI();
}
}I know you hate this, but we are not going to add tons of unholy hackery to make this "work" by some definition of work.
The only way to make this work without making a complete mess and killing performance is to do this on a best effort basis. It really does not matter whether the result of the orchestration is perfect or not. What matters is that you destroy any predictability. If the orchestration terminates early occasionally then the resulting damage is a purely academic exercise.
Making it work reliably without hard to debug livelocks and other horrors is not so academic.
So please stop overengineering this and just take the pragmatic approach of making it mostly correct. That will prevent 99.999% of realistic attacks.
It might not prevent the carefully orchestrated academic paper attack which utilizes the 0.001% failure rate by artificially enforcing a scenario which cannot be enforced in a real world environment.
Thanks,
tglx
Re: [RFC PATCH 14/16] irq: Add support for core-wide protection of IRQ and softirq
Quoted 1 line · Thomas Gleixner
Vineeth, Joel!
Hi Thomas,
Quoted 11 lines
Quoted 7 lines · Vineeth Remanan Pillai
From: "Joel Fernandes (Google)" <[email protected]>
Lastly, we also check in the schedule loop if we are about to schedule an untrusted process while the core is in such a state. This is possible if a trusted thread enters the scheduler by way of yielding CPU. This would involve no transitions through the
irq_exit()point to do any waiting, so we have to explicitly do the waiting there.Lot's of 'we' and 'this patch' in this changelog. Care to read
Documentation/process/submitting-patches.rst?
Sure, will fix it.
[..]
Quoted 15 lines
is the usual kernel coding style.
Quoted 14 lines
kernel/softirq.c +7
-0+static_branch_disable (&sched_core_irq_pause); + +return 1 ; +} +__setup ("sched_core_irq_pause=" , set_sched_core_irq_pause); +#endif + asmlinkage __visible void __softirq_entry __do_softirq(void) {unsigned long end = jiffies +MAX_SOFTIRQ_TIME ;@@ -273,6 +291,16 @@ asmlinkage __visible void __softirq_entry __do_softirq(void)/* Reset the pending bitmask before enabling irqs */ set_softirq_pending (0 );
Quoted 13 lines
kernel/softirq.c asmlinkage __visible void __softirq_entry __do_softirq(void) · line 301
Quoted 8 lines
kernel/softirq.c +8
-0+#ifdef CONFIG_SCHED_CORE_IRQ_PAUSE +/* +* Core scheduling mitigations require entry into softirq to send stall +* IPIs to sibling hyperthreads if needed (ex, sibling is running +* untrusted task). If we are here from irq_exit(), no IPIs are sent. +*/ +if (static_branch_likely (&sched_core_irq_pause)) +sched_core_irq_enter ();So you already have a #ifdef
CONFIG_SCHED_CORE_IRQ_PAUSEsection above. Why on earth can't you just provide helper functions and stubs for theCONFIG_SCHED_CORE_IRQ_PAUSE=ncase there instead of sprinkling #ifdeffery all over the place?
These ifdeffery and checkpatch / command line parameter issues were added by Vineeth before he sent out my patch. I'll let him comment on these, agreed they all need fixing!
Quoted 17 lines
kernel/softirq.c asmlinkage __visible void __softirq_entry __do_softirq(void) · line 304
Quoted 3 lines
kernel/softirq.c +2
-0+#endif +local_irq_enable ();How is
__do_softirq()relevant at all?
__do_softirq()is called from:1) the tail of interrupt handling
2) the tail oflocal_bh_enable()
3) the ksoftirq threadNone of this is relevant for what you are trying to do:
#1: Iscovered by the interrupt handling code simply because this nests
into the interrupt handling.
Quoted 6 lines
#2:
local_bh_enable()only ends up there when invoked from thread
context.This would only make sense if one sibling is in the kernel and the other in user space or guest mode. Isn't that forbidden by core scheduling in the first place?
One sibling being in the kernel, while the other sibling is in userspace is not yet forbidden by core-scheduling. We need to add support for that.
I think after reading your email and thinking about this last few days, we can make it work by sending IPIs on both syscall and IRQ entries.
Quoted 10 lines
#3: See #2
But what's worse is that this is called from an interrupt disabled region. So you brilliantly created a trivial source of livelocks.
stomp_machine()being the most obvious candidate. But it's easy enough to come up with more subtle ones.The point is that you want to bring out the sibling thread from user space or guest mode when the other HT thread:
Quoted 6 lines
A) invokes a syscall or traps into the hypervisor which is semantically equivalent (not code wise).
B) handles an interrupt/exception
So #A covers #2 and #3 above and #B covers #1 above.
Right. #A) was never implemented. But now I'm going to do it!
Quoted 2 lines
But you have the same livelock problem with doing this core wait thingy within any interrupt disabled region.
Right, which we fixed by moving the waiting code to prepare_exit_to_usermode() in the other test patch I sent to fix Aubrey's deadlock, but as you suggested I will make it work with the new generic entry code.
Quoted 4 lines
What you really want to do is on syscall and interrupt/exception entry:
if (other_sibling_in_user_or_guest()) send_IPI();
Ok.
Quoted 5 lines
Of course it's conveniant to piggy pack that onto the reschedule IPI, but that sucks. What you want is a dedicated IPI which does:
magic_ipi()set_thread_flag(TIF_CORE_SCHED_EXIT);
I tried IPIs before and ran into a few issues mostly related to CSD locking if I remember, I will try again.
Quoted 4 lines
And on the exit side you want to look at the entry code generalisation I just posted:
Thank you, I went over this code and it makes sense we work based on that.
Quoted 4 lines
Add
TIF_CORE_SCHED_EXITtoEXIT_TO_USER_MODE_WORKand toEXIT_TO_GUEST_MODE_WORKand handle it inexit_to_guest_mode_work()andexit_to_user_mode_loop()with interrupts enabled.
There's already state to determine if waiting is needed or not, we have counters that determine if any sibling HT in the core is handling an IRQ or not, and if so I wait before exiting to user mode. The outer-most IRQ entry is being tracked so we don't send IPIs on every entry.
But I got an idea from your email. Whenever a 'tagged' task becomes current, I can set a TIF flag marking that it is a tagged task and needs special handling when exiting to user mode (or guest). Then on exiting to usermode (or guest), I can check if any waiting is needed and wait there in the interrupt enabled regions you are suggesting. This should simplify the code since currently we have that complex if expression to determine if the task is "special".
Quoted 11 lines
Trying to do this in fully interrupt disabled context is just a recipe for disaster.
The entry case condition wants to have a TIF bit as well, i.e.
if (thread_test_bit(TIF_CORE_SCHED_REQUIRED) { sched_ipi_dance() { if (other_sibling_in_user_or_guest()) send_IPI(); } }
I did not understand this bit. Could you explain more about it? Are you talking about the IPIs sent from the schedule() loop in this series?
Quoted 10 lines
I know you hate this, but we are not going to add tons of unholy hackery to make this "work" by some definition of work.
The only way to make this work without making a complete mess and killing performance is to do this on a best effort basis. It really does not matter whether the result of the orchestration is perfect or not. What matters is that you destroy any predictability. If the orchestration terminates early occasionally then the resulting damage is a purely academic exercise.
Quoted 6 lines
Making it work reliably without hard to debug livelocks and other horrors is not so academic.
So please stop overengineering this and just take the pragmatic approach of making it mostly correct. That will prevent 99.999% of realistic attacks.
Agreed. Actually that's what I was aiming for. Even the IPIs are sent only if needed. I did a lot of benchmarking and did not see a performance hit.
Quoted 4 lines
It might not prevent the carefully orchestrated academic paper attack which utilizes the 0.001% failure rate by artificially enforcing a scenario which cannot be enforced in a real world environment.
Got it.
thanks,
- Joel
Re: [RFC PATCH 14/16] irq: Add support for core-wide protection of IRQ and softirq
Joel,
Quoted 11 lines · Joel Fernandes
Quoted 9 lines · Thomas Gleixner
The entry case condition wants to have a TIF bit as well, i.e.
if (thread_test_bit(TIF_CORE_SCHED_REQUIRED) { sched_ipi_dance() { if (other_sibling_in_user_or_guest()) send_IPI(); } }
Quoted 2 lines
I did not understand this bit. Could you explain more about it? Are you talking about the IPIs sent from the
schedule()loop in this series?
Nah, let me try again. If two tasks are out in user space (or guest mode) and they fall under the isolation rule that they either are both in user space or both in the kernel then you tag both with TIF_CORE_SCHED_REQUIRED or whatever bit is appropriate.
So in entry from user you do:
if (thread_test_bit(TIF_CORE_SCHED_REQUIRED)) sched_orchestrate_entry();
void sched_orchestrate_entry(void)
{
if (other_sibling_in_user_or_guest())
send_IPI_to_sibling();
}That IPI brings the sibling out of user or guest mode.
On the way back to user/guest you do:
if (thread_test_bit(TIF_CORE_SCHED_REQUIRED)) sched_orchestrate_exit();
void sched_orchestrate_exit(void)
{
while (other_sibling_in_kernel())
twiddle_thumbs();
}Hope that clarifies it.
Thanks,
tglx
Re: [RFC PATCH 14/16] irq: Add support for core-wide protection of IRQ and softirq
In reply to Joel FernandesQuoted 5 lines · Joel Fernandes
These ifdeffery and checkpatch / command line parameter issues were added by Vineeth before he sent out my patch. I'll let him comment on these, agreed they all need fixing!
Will fix this in the next iteration. Regarding the __setup code, I shamelessly copied it from page_alloc.c during development and missed reviewing it before sending it out. All valid points and will fix it with the next iteration.
Thanks,
Vineeth
RFC PATCH 15/16 Documentation: Add documentation on core scheduling
In reply to Vineeth Remanan PillaiFrom: "Joel Fernandes (Google)" <[email protected]>
--- .../admin-guide/hw-vuln/core-scheduling.rst | 241 ++++++++++++++++++ Documentation/admin-guide/hw-vuln/index.rst | 1 + 2 files changed, 242 insertions(+) create mode 100644 Documentation/admin-guide/hw-vuln/core-scheduling.rst
Documentation/admin-guide/hw-vuln/core-scheduling.rst new file +241 -0
@@ -0,0 +1,241 @@ +Core Scheduling +================ +MDS and L1TF mitigations do not protect from cross-HT attacks (attacker running +on one HT with victim running on another). For proper mitigation of this, +core scheduling support is available via the ``CONFIG_SCHED_CORE`` config option. +Using this feature, userspace defines groups of tasks that trust each other. +The core scheduler uses this information to make sure that tasks that do not +trust each other will never run simultaneously on a core while ensuring trying +to maintain and ensure scheduler properties and requirements. + +Usage +----- +The user-interface to this feature is not yet finalized. CUrrent implementation +uses CPU controller cgroup. Core scheduling adds a ``cpu.tag`` file to the CPU +controller CGroup. If the content of this file is 1, then all the tasks in this +CGroup trust each other and are allowed to run concurrently on the siblings of +a core. + +This interface has drawbacks. Trusted tasks has to be grouped into one CPU CGroup +and this is not always possible based on system's existing Cgroup configuration, +where trusted tasks could already be in different CPU Cgroups. Also, this feature +will have a hard dependency on CGroups and systems with CGroups disabled would not +be able to use core scheduling. See `Future work`_ for other API proposals. + +Design/Implementation +--------------------- +Tasks are grouped as mentioned in `Usage`_ and tasks that trust each other +share the same cookie value(in task_struct). + +The basic idea is that, every schedule event tries to select tasks for all the +siblings of a core such that all the selected tasks are trusted(same cookie). + +During a schedule event on any sibling of a core, the highest priority task for +that core is picked and assigned to the sibling which has it enqueued. For rest of +the siblings in the core, highest priority task with the same cookie is selected if +there is one runnable in the run queue. If a task with same cookie is not available, +idle task is selected. Idle task is globally trusted. + +Once a task has been selected for all the siblings in the core, an IPI is sent to +all the siblings who has a new task selected. Siblings on receiving the IPI, will +switch to the new task immediately. + +Force-idling of tasks +--------------------- +The scheduler tries its best to find tasks that trust each other such that all +tasks selected to be scheduled are of the highest priority in that runqueue. +However, this is not always possible. Favoring security over fairness, one or +more siblings could be forced to select a lower priority task if the highest +priority task is not trusted with respect to the core wide highest priority task. +If a sibling does not have a trusted task to run, it will be forced idle by the +scheduler(idle thread is scheduled to run). + +When the highest priorty task is selected to run, a reschedule-IPI is sent to +the sibling to force it into idle. This results in 4 cases which need to be +considered depending on whether a VM or a regular usermode process was running +on either HT: + +:: + HT1 (attack) HT2 (victim) + + A idle -> user space user space -> idle + + B idle -> user space guest -> idle + + C idle -> guest user space -> idle + + D idle -> guest guest -> idle + +Note that for better performance, we do not wait for the destination CPU +(victim) to enter idle mode. This is because the sending of the IPI would +bring the destination CPU immediately into kernel mode from user space, or +VMEXIT from guests. At best, this would only leak some scheduler metadata which +may not be worth protecting. + +Protection against interrupts using IRQ pausing +----------------------------------------------- +The scheduler on its own cannot protect interrupt data. This is because the +scheduler is unaware of interrupts at scheduling time. To mitigate this, we +send an IPI to siblings on IRQ entry. This IPI handler busy-waits until the IRQ +on the sending HT exits. For good performance, we send an IPI only if it is +detected that the core is running tasks that have been marked for +core scheduling. Both interrupts and softirqs are protected. + +This protection can be disabled by disabling ``CONFIG_SCHED_CORE_IRQ_PAUSE`` or +through the ``sched_core_irq_pause`` boot parameter. + +If it is desired to disable IRQ pausing, other mitigation methods can be considered: + +1. Changing interrupt affinities to a trusted core which does not execute untrusted tasks +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +By changing the interrupt affinities to a designated safe-CPU which runs +only trusted tasks, IRQ data can be protected. One issue is this involves +giving up a full CPU core of the system to run safe tasks. Another is that, +per-cpu interrupts such as the local timer interrupt cannot have their +affinity changed. also, sensitive timer callbacks such as the random entropy timer +can run in softirq on return from these interrupts and expose sensitive +data. In the future, that could be mitigated by forcing softirqs into threaded +mode by utilizing a mechanism similar to ``PREEMPT_RT``. + +Yet another issue with this is, for multiqueue devices with managed +interrupts, the IRQ affinities cannot be changed however it could be +possible to force a reduced number of queues which would in turn allow to +shield one or two CPUs from such interrupts and queue handling for the price +of indirection. + +2. Running IRQs as threaded-IRQs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +This would result in forcing IRQs into the scheduler which would then provide +the process-context mitigation. However, not all interrupts can be threaded. + +Trust model +----------- +Core scheduling understands trust relationships by assignment of a cookie to +related tasks using the above mentioned interface. When a system with core +scheduling boots, all tasks are considered to trust each other. This is because +the scheduler does not have information about trust relationships. That is, all +tasks have a default cookie value of 0. This cookie value is also considered +the system-wide cookie value and the IRQ-pausing mitigation is avoided if +siblings are running these cookie-0 tasks. + +By default, all system processes on boot are considered trusted and userspace +has to explicitly use the interfaces mentioned above to group sets of tasks. +Tasks within the group trust each other, but not those outside. Tasks outside +the group don't trust the task inside. + +Limitations +----------- +Core scheduling tries to guarentee that only trusted tasks run concurrently on a +core. But there could be small window of time during which untrusted tasks run +concurrently or kernel could be running concurrently with a task not trusted by +kernel. + +1. IPI processing delays +^^^^^^^^^^^^^^^^^^^^^^^^ +Core scheduling selects only trusted tasks to run together. IPI is used to notify +the siblings to switch to the new task. But there could be hardware delays in +receiving of the IPI on some arch (on x86, this has not been observed). This may +cause an attacker task to start running on a cpu before its siblings receive the +IPI. Even though cache is flushed on entry to user mode, victim tasks on siblings +may populate data in the cache and micro acrhitectural buffers after the attacker +starts to run and this is a possibility for data leak. + +2. Asynchronous Kernel entries +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +A task can switch to kernel any time due to events like irqs, system calls etc. +Since core scheduling synchronizes only during a schedule event, kernel can run +along with a task that it doesn't trust. The IRQ pause mechanism mentioned above, +provides protection during nmi/irq/softirqs. But tasks could still enter kernel +mode via system calls and this is not currently protected. + +There are ideas about mitigating this: + - Kernel Address Space Isolation: System calls could run in a much restricted + address space which is guarenteed not to leak any sensitive data. There are + practical limitation in implementing this - the main concern being how to + decided on an address space that is guarenteed to not have any sensitive + data + - On a system call, change the cookie to the system trusted cookie and initiate + a schedule event. This would be better than pausing all the siblings during + the entire duration for the system call, but still would be a huge hit to the + performance. + +Open cross-HT issues that core scheduling does not solve +-------------------------------------------------------- +1. For MDS +^^^^^^^^^^ +Core scheduling cannot protect against MDS attacks between an HT running in +user mode and another running in kernel mode. Even though both HTs run tasks +which trust each other, kernel memory is still considered untrusted. Such +attacks are possible for any combination of sibling CPU modes (host or guest mode). + +2. For L1TF +^^^^^^^^^^^ +Core scheduling cannot protect against a L1TF guest attackers exploiting a +guest or host victim. This is because the guest attacker can craft invalid +PTEs which are not inverted due to a vulnerable guest kernel. The only +solution is to disable EPT. + +For both MDS and L1TF, if the guest vCPU is configured to not trust each +other (by tagging separately), then the guest to guest attacks would go away. +Or it could be a system admin policy which considers guest to guest attacks as +a guest problem. + +Another approach to resolve these would be to make every untrusted task on the +system to not trust every other untrusted task. While this could reduce +parallelism of the untrusted tasks, it would still solve the above issues while +allowing system processes (trusted tasks) to share a core. + +Use cases +--------- +The main use case for Core scheduling is mitigating the cross-HT vulnerabilities +with SMT enabled. There are other use cases where this feature could be used: + +- Isolating tasks that needs a whole core: Examples include realtime tasks, tasks + that uses SIMD instructions etc. +- Gang scheduling: Requirements for a group of tasks that needs to be scheduled + together could also be realized using core scheduling. One example is vcpus of + a VM. + +Future work +----------- +1. API Proposals +^^^^^^^^^^^^^^^^ + +As mentioned in `Usage`_ section, various API proposals are listed here: + +- ``prctl`` : We can pass in a tag and all tasks with same tag set by prctl forms + a trusted group. + +- ``sched_setattr`` : Similar to prctl, but has the advantage that tasks could be + tagged by other tasks with appropriate permissions. + +- ``Auto Tagging`` : Related tasks are tagged automatically. Relation could be, + threads of the same process, tasks by a user, group or session etc. + +- Dedicated cgroup or procfs/sysfs interface for grouping trusted tasks. This could + be combined with prctl/sched_setattr as well. + +2. Auto-tagging of KVM vCPU threads +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +To make configuration easier, it would be great if KVM auto-tags vCPU threads +such that a given VM only trusts other vCPUs of the same VM. Or something more +aggressive like assiging a vCPU thread a unique tag. + +3. Auto-tagging of processes by default +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Currently core scheduling does not prevent 'unconfigured' tasks from being +co-scheduled on the same core. In other words, everything trusts everything +else by default. If a user wants everything default untrusted, a CONFIG option +could be added to assign every task with a unique tag by default. + +4. Auto-tagging on fork +^^^^^^^^^^^^^^^^^^^^^^^ +Currently, on fork a thread is added to the same trust-domain as the parent. For +systems which want all tasks to have a unique tag, it could be desirable to assign +a unique tag to a task so that the parent does not trust the child by default. + +5. Skipping per-HT mitigations if task is trusted +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +If core scheduling is enabled, by default all tasks trust each other as +mentioned above. In such scenario, it may be desirable to skip the same-HT +mitigations on return to the trusted user-mode to improve performance.
Documentation/admin-guide/hw-vuln/index.rst +1 -0
@@ -15,3 +15,4 @@ are configurable at compile, boot or run time. tsx_async_abort multihit.rst special-register-buffer-data-sampling.rst + core-scheduling.rst
Signature · 1 line
2.17.1
RFC PATCH 16/16 sched: Debug bits...
In reply to Vineeth Remanan PillaiFrom: Peter Zijlstra <[email protected]>
--- kernel/sched/core.c | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-)
kernel/sched/core.c +42 -2
@@ -105,6 +105,10 @@ static inline bool prio_less(struct task_struct *a, struct task_struct *b)int pa =__task_prio (a), pb =__task_prio (b); +trace_printk ("(%s/%d;%d,%Lu,%Lu) ?< (%s/%d;%d,%Lu,%Lu)\n" , + a->comm, a->pid, pa, a->se.vruntime, a->dl.deadline, + b->comm, b->pid, pb, b->se.vruntime, b->dl.deadline); +if (-pa < -pb)return true;
@@ -302,12 +306,16 @@ static void __sched_core_enable(void)static_branch_enable (&__sched_core_enabled);stop_machine (__sched_core_stopper, (void *)true,NULL ); + +printk ("core sched enabled\n" ); }static void __sched_core_disable (void ) {stop_machine (__sched_core_stopper, (void *)false,NULL );static_branch_disable (&__sched_core_enabled); + +printk ("core sched disabled\n" ); }void sched_core_get (void )
@@ -4477,6 +4485,14 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)put_prev_task (rq, prev);set_next_task (rq, next); } + +trace_printk ("pick pre selected (%u %u %u): %s/%d %lx\n" , + rq->core->core_task_seq, + rq->core->core_pick_seq, + rq->core_sched_seq, + next->comm, next->pid, + next->core_cookie); +return next; }
@@ -4551,6 +4567,9 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)*/ if (i == cpu && !need_sync && !p->core_cookie) { next = p; +trace_printk ("unconstrained pick: %s/%d %lx\n" , + next->comm, next->pid, next->core_cookie); + goto done; }
@@ -4559,6 +4578,9 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) rq_i->core_pick = p; +trace_printk ("cpu(%d): selected: %s/%d %lx\n" , + i, p->comm, p->pid, p->core_cookie); +/* * If this new candidate is of higher priority than the * previous; and they're incompatible; we need to wipe
@@ -4575,6 +4597,8 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) rq->core->core_cookie = p->core_cookie; max = p; +trace_printk ("max: %s/%d %lx\n" , max->comm, max->pid, max->core_cookie); +if (old_max) {for_each_cpu (j, smt_mask) {if (j == i)
@@ -4602,6 +4626,7 @@ next_class:; rq->core->core_pick_seq = rq->core->core_task_seq; next = rq->core_pick; rq->core_sched_seq = rq->core->core_pick_seq; +trace_printk ("picked: %s/%d %lx\n" , next->comm, next->pid, next->core_cookie);/* * Reschedule siblings
@@ -4624,11 +4649,20 @@ next_class:;if (i == cpu)continue ; -if (rq_i->curr != rq_i->core_pick) +if (rq_i->curr != rq_i->core_pick) { +trace_printk ("IPI(%d)\n" , i);resched_curr (rq_i); + }/* Did we break L1TF mitigation requirements? */ -WARN_ON_ONCE (!cookie_match (next, rq_i->core_pick)); +if (unlikely (!cookie_match (next, rq_i->core_pick))) { +trace_printk ("[%d]: cookie mismatch. %s/%d/0x%lx/0x%lx\n" , + rq_i->cpu, rq_i->core_pick->comm, + rq_i->core_pick->pid, + rq_i->core_pick->core_cookie, + rq_i->core->core_cookie); +WARN_ON_ONCE (1 ); + } } done:
@@ -4667,6 +4701,10 @@ static bool try_steal_cookie(int this, int that)if (p->core_occupation > dst->idle->core_occupation) goto next; +trace_printk ("core fill: %s/%d (%d->%d) %d %d %lx\n" , + p->comm, p->pid, that, this, + p->core_occupation, dst->idle->core_occupation, cookie); + p->on_rq =TASK_ON_RQ_MIGRATING ;deactivate_task (src, p,0 );set_task_cpu (p, this);
@@ -7305,6 +7343,8 @@ int sched_cpu_starting(unsigned int cpu)WARN_ON_ONCE (rq->core && rq->core != core_rq); rq->core = core_rq; } + +printk ("core: %d -> %d\n" , cpu,cpu_of (core_rq));#endif /* CONFIG_SCHED_CORE */ sched_rq_cpu_starting (cpu);
Signature · 1 line
2.17.1
Quoted 4 lines · Vineeth Pillai
Sixth iteration of the Core-Scheduling feature.
I am no longer with DigitalOcean. Kindly use this email address for all future responses.
Thanks,
Vineeth
Quoted 32 lines · Vineeth Remanan Pillai
Sixth iteration of the Core-Scheduling feature.
Core scheduling is a feature that allows only trusted tasks to run concurrently on cpus sharing compute resources (eg: hyperthreads on a core). The goal is to mitigate the core-level side-channel attacks without requiring to disable SMT (which has a significant impact on performance in some situations). Core scheduling (as of v6) mitigates user-space to user-space attacks and user to kernel attack when one of the siblings enters the kernel via interrupts. It is still possible to have a task attack the sibling thread when it enters the kernel via syscalls.
By default, the feature doesn't change any of the current scheduler behavior. The user decides which tasks can run simultaneously on the same core (for now by having them in the same tagged cgroup). When a tag is enabled in a cgroup and a task from that cgroup is running on a hardware thread, the scheduler ensures that only idle or trusted tasks run on the other sibling(s). Besides security concerns, this feature can also be beneficial for RT and performance applications where we want to control how tasks make use of SMT dynamically.
This iteration is mostly a cleanup of v5 except for a major feature of pausing sibling when a cpu enters kernel via nmi/irq/softirq. Also introducing documentation and includes minor crash fixes.
One major cleanup was removing the hotplug support and related code. The hotplug related crashes were not documented and the fixes piled up over time leading to complex code. We were not able to reproduce the crashes in the limited testing done. But if they are reroducable, we don't want to hide them. We should document them and design better fixes if any.
Quoted 15 lines
In terms of performance, the results in this release are similar to v5. On a x86 system with N hardware threads:
- if only N/2 hardware threads are busy, the performance is similar between baseline, corescheduling and nosmt
- if N hardware threads are busy with N different corescheduling groups, the impact of corescheduling is similar to nosmt
- if N hardware threads are busy and multiple active threads share the same corescheduling cookie, they gain a performance improvement over nosmt.
The specific performance impact depends on the workload, but for a really busy database 12-vcpu VM (1 coresched tag) running on a 36 hardware threads NUMA node with 96 mostly idle neighbor VMs (each in their own coresched tag), the performance drops by 54% with corescheduling and drops by 90% with nosmt.
We found uperf(in cgroup) throughput drops by ~50% with corescheduling.
The problem is, uperf triggered a lot of softirq and offloaded softirq service to *ksoftirqd* thread.
- default, ksoftirqd thread can run with uperf on the same core, we saw 100% CPU utilization.
- coresched enabled, ksoftirqd's core cookie is different from uperf, so they can't run concurrently on the same core, we saw ~15% forced idle.
I guess this kind of performance drop can be replicated by other similar (a lot of softirq activities) workloads.
Currently core scheduler picks cookie-match tasks for all SMT siblings, does it make sense we add a policy to allow cookie-compatible task running together? For example, if a task is trusted(set by admin), it can work with kernel thread. The difference from corescheduling disabled is that we still have user to user isolation.
Thanks,
-Aubrey
Hi Aubrey,
Quoted 60 lines · Aubrey
Quoted 32 lines · Vineeth Remanan Pillai
Sixth iteration of the Core-Scheduling feature.
Core scheduling is a feature that allows only trusted tasks to run concurrently on cpus sharing compute resources (eg: hyperthreads on a core). The goal is to mitigate the core-level side-channel attacks without requiring to disable SMT (which has a significant impact on performance in some situations). Core scheduling (as of v6) mitigates user-space to user-space attacks and user to kernel attack when one of the siblings enters the kernel via interrupts. It is still possible to have a task attack the sibling thread when it enters the kernel via syscalls.
By default, the feature doesn't change any of the current scheduler behavior. The user decides which tasks can run simultaneously on the same core (for now by having them in the same tagged cgroup). When a tag is enabled in a cgroup and a task from that cgroup is running on a hardware thread, the scheduler ensures that only idle or trusted tasks run on the other sibling(s). Besides security concerns, this feature can also be beneficial for RT and performance applications where we want to control how tasks make use of SMT dynamically.
This iteration is mostly a cleanup of v5 except for a major feature of pausing sibling when a cpu enters kernel via nmi/irq/softirq. Also introducing documentation and includes minor crash fixes.
One major cleanup was removing the hotplug support and related code. The hotplug related crashes were not documented and the fixes piled up over time leading to complex code. We were not able to reproduce the crashes in the limited testing done. But if they are reroducable, we don't want to hide them. We should document them and design better fixes if any.
Quoted 15 lines
In terms of performance, the results in this release are similar to v5. On a x86 system with N hardware threads:
- if only N/2 hardware threads are busy, the performance is similar between baseline, corescheduling and nosmt
- if N hardware threads are busy with N different corescheduling groups, the impact of corescheduling is similar to nosmt
- if N hardware threads are busy and multiple active threads share the same corescheduling cookie, they gain a performance improvement over nosmt.
The specific performance impact depends on the workload, but for a really busy database 12-vcpu VM (1 coresched tag) running on a 36 hardware threads NUMA node with 96 mostly idle neighbor VMs (each in their own coresched tag), the performance drops by 54% with corescheduling and drops by 90% with nosmt.We found uperf(in cgroup) throughput drops by ~50% with corescheduling.
The problem is, uperf triggered a lot of softirq and offloaded softirq service to *ksoftirqd* thread.
- default, ksoftirqd thread can run with uperf on the same core, we saw 100% CPU utilization.
- coresched enabled, ksoftirqd's core cookie is different from uperf, so they can't run concurrently on the same core, we saw ~15% forced idle.
Quoted 8 lines
I guess this kind of performance drop can be replicated by other similar (a lot of softirq activities) workloads.
Currently core scheduler picks cookie-match tasks for all SMT siblings, does it make sense we add a policy to allow cookie-compatible task running together? For example, if a task is trusted(set by admin), it can work with kernel thread. The difference from corescheduling disabled is that we still have user to user isolation.
In ChromeOS we are considering all cookie-0 tasks as trusted. Basically if you don't trust a task, then that is when you assign the task a tag. We do this for the sandboxed processes.
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Quoted 72 lines · Joel Fernandes
Hi Aubrey,
Quoted 60 lines · Aubrey
Quoted 32 lines · Vineeth Remanan Pillai
Sixth iteration of the Core-Scheduling feature.
Core scheduling is a feature that allows only trusted tasks to run concurrently on cpus sharing compute resources (eg: hyperthreads on a core). The goal is to mitigate the core-level side-channel attacks without requiring to disable SMT (which has a significant impact on performance in some situations). Core scheduling (as of v6) mitigates user-space to user-space attacks and user to kernel attack when one of the siblings enters the kernel via interrupts. It is still possible to have a task attack the sibling thread when it enters the kernel via syscalls.
By default, the feature doesn't change any of the current scheduler behavior. The user decides which tasks can run simultaneously on the same core (for now by having them in the same tagged cgroup). When a tag is enabled in a cgroup and a task from that cgroup is running on a hardware thread, the scheduler ensures that only idle or trusted tasks run on the other sibling(s). Besides security concerns, this feature can also be beneficial for RT and performance applications where we want to control how tasks make use of SMT dynamically.
This iteration is mostly a cleanup of v5 except for a major feature of pausing sibling when a cpu enters kernel via nmi/irq/softirq. Also introducing documentation and includes minor crash fixes.
One major cleanup was removing the hotplug support and related code. The hotplug related crashes were not documented and the fixes piled up over time leading to complex code. We were not able to reproduce the crashes in the limited testing done. But if they are reroducable, we don't want to hide them. We should document them and design better fixes if any.
Quoted 15 lines
In terms of performance, the results in this release are similar to v5. On a x86 system with N hardware threads:
- if only N/2 hardware threads are busy, the performance is similar between baseline, corescheduling and nosmt
- if N hardware threads are busy with N different corescheduling groups, the impact of corescheduling is similar to nosmt
- if N hardware threads are busy and multiple active threads share the same corescheduling cookie, they gain a performance improvement over nosmt.
The specific performance impact depends on the workload, but for a really busy database 12-vcpu VM (1 coresched tag) running on a 36 hardware threads NUMA node with 96 mostly idle neighbor VMs (each in their own coresched tag), the performance drops by 54% with corescheduling and drops by 90% with nosmt.We found uperf(in cgroup) throughput drops by ~50% with corescheduling.
The problem is, uperf triggered a lot of softirq and offloaded softirq service to *ksoftirqd* thread.
- default, ksoftirqd thread can run with uperf on the same core, we saw 100% CPU utilization.
- coresched enabled, ksoftirqd's core cookie is different from uperf, so they can't run concurrently on the same core, we saw ~15% forced idle.Quoted 8 lines
I guess this kind of performance drop can be replicated by other similar (a lot of softirq activities) workloads.
Currently core scheduler picks cookie-match tasks for all SMT siblings, does it make sense we add a policy to allow cookie-compatible task running together? For example, if a task is trusted(set by admin), it can work with kernel thread. The difference from corescheduling disabled is that we still have user to user isolation.
Quoted 3 lines
In ChromeOS we are considering all cookie-0 tasks as trusted. Basically if you don't trust a task, then that is when you assign the task a tag. We do this for the sandboxed processes.
I have a proposal of this, by changing cpu.tag to cpu.coresched_policy, something like the following:
+/*
+ * Core scheduling policy:
+ * - CORE_SCHED_DISABLED: core scheduling is disabled.
+ * - CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * - CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * - CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */
+enum coresched_policy {
+ CORE_SCHED_DISABLED,
+ CORE_SCHED_COOKIE_MATCH,
+ CORE_SCHED_COOKIE_TRUST,
+ CORE_SCHED_COOKIE_LONELY,
+};
We can set policy to CORE_COOKIE_TRUST of uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?
Quoted 2 lines
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
This is a good question, from the data we measured by uperf, SMT+core-scheduling is 28.2% worse than no-SMT, :(
Thanks,
-Aubrey
Quoted 9 lines
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Re: [RFC PATCH 00/16] Core scheduling v6(Internet mail)
Hi,
Quoted 82 lines · Aubrey
Quoted 72 lines · Joel Fernandes
Hi Aubrey,
Quoted 60 lines · Aubrey
Quoted 32 lines · Vineeth Remanan Pillai
Sixth iteration of the Core-Scheduling feature.
Core scheduling is a feature that allows only trusted tasks to run concurrently on cpus sharing compute resources (eg: hyperthreads on a core). The goal is to mitigate the core-level side-channel attacks without requiring to disable SMT (which has a significant impact on performance in some situations). Core scheduling (as of v6) mitigates user-space to user-space attacks and user to kernel attack when one of the siblings enters the kernel via interrupts. It is still possible to have a task attack the sibling thread when it enters the kernel via syscalls.
By default, the feature doesn't change any of the current scheduler behavior. The user decides which tasks can run simultaneously on the same core (for now by having them in the same tagged cgroup). When a tag is enabled in a cgroup and a task from that cgroup is running on a hardware thread, the scheduler ensures that only idle or trusted tasks run on the other sibling(s). Besides security concerns, this feature can also be beneficial for RT and performance applications where we want to control how tasks make use of SMT dynamically.
This iteration is mostly a cleanup of v5 except for a major feature of pausing sibling when a cpu enters kernel via nmi/irq/softirq. Also introducing documentation and includes minor crash fixes.
One major cleanup was removing the hotplug support and related code. The hotplug related crashes were not documented and the fixes piled up over time leading to complex code. We were not able to reproduce the crashes in the limited testing done. But if they are reroducable, we don't want to hide them. We should document them and design better fixes if any.
Quoted 15 lines
In terms of performance, the results in this release are similar to v5. On a x86 system with N hardware threads:
- if only N/2 hardware threads are busy, the performance is similar between baseline, corescheduling and nosmt
- if N hardware threads are busy with N different corescheduling groups, the impact of corescheduling is similar to nosmt
- if N hardware threads are busy and multiple active threads share the same corescheduling cookie, they gain a performance improvement over nosmt.
The specific performance impact depends on the workload, but for a really busy database 12-vcpu VM (1 coresched tag) running on a 36 hardware threads NUMA node with 96 mostly idle neighbor VMs (each in their own coresched tag), the performance drops by 54% with corescheduling and drops by 90% with nosmt.We found uperf(in cgroup) throughput drops by ~50% with corescheduling.
The problem is, uperf triggered a lot of softirq and offloaded softirq service to *ksoftirqd* thread.
- default, ksoftirqd thread can run with uperf on the same core, we saw 100% CPU utilization.
- coresched enabled, ksoftirqd's core cookie is different from uperf, so they can't run concurrently on the same core, we saw ~15% forced idle.Quoted 8 lines
I guess this kind of performance drop can be replicated by other similar (a lot of softirq activities) workloads.
Currently core scheduler picks cookie-match tasks for all SMT siblings, does it make sense we add a policy to allow cookie-compatible task running together? For example, if a task is trusted(set by admin), it can work with kernel thread. The difference from corescheduling disabled is that we still have user to user isolation.
Quoted 3 lines
In ChromeOS we are considering all cookie-0 tasks as trusted. Basically if you don't trust a task, then that is when you assign the task a tag. We do this for the sandboxed processes.
I have a proposal of this, by changing cpu.tag to cpu.coresched_policy, something like the following:
Quoted 7 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
How about other OS tasks(like systemd) except kernel thread? :)
Thx.
Regards,
Jiang
Quoted 30 lines
+ * -
CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Quoted 2 lines
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
This is a good question, from the data we measured by uperf, SMT+core-scheduling is 28.2% worse than no-SMT, :(
Thanks,
-AubreyQuoted 8 lines
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]
Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};
Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?
Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
5. Document everything clearly so the semantics are clear both to the developers of core scheduling and to system administrators.
Note that, with the concept of "system trusted cookie", we can also do optimizations like:
1. Disable STIBP when switching into trusted tasks.
2. Disable L1D flushing / verw stuff for L1TF/MDS issues, when switching into trusted tasks.
At least #1 seems to be biting enabling HT on ChromeOS right now, and one other engineer requested I do something like #2 already.
Once we get full-syscall isolation working, threads belonging to a process can also share a core so those can just share a core with the task-group leader.
Quoted 4 lines
Quoted 1 line
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
This is a good question, from the data we measured by uperf, SMT+core-scheduling is 28.2% worse than no-SMT, :(
This is worrying for sure. :-(. We ought to debug/profile it more to see what is causing the overhead. Me/Vineeth added it as a topic for LPC as well.
Any other thoughts from others on this?
thanks,
- Joel
Quoted 9 lines
Quoted 8 lines
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Thanks,
-Aubrey
Quoted 41 lines
5. Document everything clearly so the semantics are clear both to the developers of core scheduling and to system administrators.
Note that, with the concept of "system trusted cookie", we can also do optimizations like:
1. Disable STIBP when switching into trusted tasks.
2. Disable L1D flushing / verw stuff for L1TF/MDS issues, when switching into trusted tasks.At least #1 seems to be biting enabling HT on ChromeOS right now, and one other engineer requested I do something like #2 already.
Once we get full-syscall isolation working, threads belonging to a process can also share a core so those can just share a core with the task-group leader.
Quoted 4 lines
Quoted 1 line
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
This is a good question, from the data we measured by uperf, SMT+core-scheduling is 28.2% worse than no-SMT, :(
This is worrying for sure. :-(. We ought to debug/profile it more to see what is causing the overhead. Me/Vineeth added it as a topic for LPC as well.
Any other thoughts from others on this?
thanks,
- Joel
Quoted 9 lines
Quoted 8 lines
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Quoted 45 lines · Aubrey
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).
Quoted 5 lines
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Yes, but let me try to understand better. There are 2 problems here I think:
1. ksoftirqd getting idled when HT is turned on, because uperf is sharing a core with it: This should not be any worse than SMT OFF, because even SMT OFF would also reduce ksoftirqd's CPU time just core sched is doing. Sure core-scheduling adds some overhead with IPIs but such a huge drop of perf is strange. Peter any thoughts on that?
2. Interface: To solve the performance problem, you are saying you want uperf to share a core with ksoftirqd so that it is not forced into idle. Why not just keep uperf out of the cgroup? Then it will have cookie 0 and be able to share core with kernel threads. About user-user isolation that you need, if you tag any "untrusted" threads by adding it to CGroup, then there will automatically isolated from uperf while allowing uperf to share CPU with kernel threads.
Please let me know your thoughts and thanks,
- Joel
Quoted 45 lines
Thanks,
-AubreyQuoted 41 lines
5. Document everything clearly so the semantics are clear both to the developers of core scheduling and to system administrators.
Note that, with the concept of "system trusted cookie", we can also do optimizations like:
1. Disable STIBP when switching into trusted tasks.
2. Disable L1D flushing / verw stuff for L1TF/MDS issues, when switching into trusted tasks.At least #1 seems to be biting enabling HT on ChromeOS right now, and one other engineer requested I do something like #2 already.
Once we get full-syscall isolation working, threads belonging to a process can also share a core so those can just share a core with the task-group leader.
Quoted 4 lines
Quoted 1 line
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
This is a good question, from the data we measured by uperf, SMT+core-scheduling is 28.2% worse than no-SMT, :(
This is worrying for sure. :-(. We ought to debug/profile it more to see what is causing the overhead. Me/Vineeth added it as a topic for LPC as well.
Any other thoughts from others on this?
thanks,
- Joel
Quoted 9 lines
Quoted 8 lines
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Quoted 60 lines · Joel Fernandes
Quoted 45 lines · Aubrey
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).Quoted 5 lines
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Yes, but let me try to understand better. There are 2 problems here I think:
1. ksoftirqd getting idled when HT is turned on, because uperf is sharing a core with it: This should not be any worse than SMT OFF, because even SMT OFF would also reduce ksoftirqd's CPU time just core sched is doing. Sure core-scheduling adds some overhead with IPIs but such a huge drop of perf is strange. Peter any thoughts on that?
Quoted 3 lines
2. Interface: To solve the performance problem, you are saying you want uperf to share a core with ksoftirqd so that it is not forced into idle. Why not just keep uperf out of the cgroup?
I guess this is unacceptable for who runs their apps in container and vm.
Thanks,
-Aubrey
Quoted 55 lines
Then it will have cookie 0 and be able to
share core with kernel threads. About user-user isolation that you need, if you tag any "untrusted" threads by adding it to CGroup, then there will automatically isolated from uperf while allowing uperf to share CPU with kernel threads.Please let me know your thoughts and thanks,
- Joel
Quoted 45 lines
Thanks,
-AubreyQuoted 41 lines
5. Document everything clearly so the semantics are clear both to the developers of core scheduling and to system administrators.
Note that, with the concept of "system trusted cookie", we can also do optimizations like:
1. Disable STIBP when switching into trusted tasks.
2. Disable L1D flushing / verw stuff for L1TF/MDS issues, when switching into trusted tasks.At least #1 seems to be biting enabling HT on ChromeOS right now, and one other engineer requested I do something like #2 already.
Once we get full-syscall isolation working, threads belonging to a process can also share a core so those can just share a core with the task-group leader.
Quoted 4 lines
Quoted 1 line
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
This is a good question, from the data we measured by uperf, SMT+core-scheduling is 28.2% worse than no-SMT, :(
This is worrying for sure. :-(. We ought to debug/profile it more to see what is causing the overhead. Me/Vineeth added it as a topic for LPC as well.
Any other thoughts from others on this?
thanks,
- Joel
Quoted 9 lines
Quoted 8 lines
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Re: [RFC PATCH 00/16] Core scheduling v6(Internet mail)
Quoted 65 lines · Aubrey
Quoted 60 lines · Joel Fernandes
Quoted 45 lines · Aubrey
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).Quoted 5 lines
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Yes, but let me try to understand better. There are 2 problems here I think:
1. ksoftirqd getting idled when HT is turned on, because uperf is sharing a core with it: This should not be any worse than SMT OFF, because even SMT OFF would also reduce ksoftirqd's CPU time just core sched is doing. Sure core-scheduling adds some overhead with IPIs but such a huge drop of perf is strange. Peter any thoughts on that?
Quoted 5 lines
Quoted 3 lines
2. Interface: To solve the performance problem, you are saying you want uperf to share a core with ksoftirqd so that it is not forced into idle. Why not just keep uperf out of the cgroup?
I guess this is unacceptable for who runs their apps in container and vm.
IMHO, just as Joel proposed,
1. Consider all cookie-0 task as trusted.
2. All kernel threads and idle task would have a cookie 0
In that way, all tasks with cookies(including uperf in a cgroup) could run concurrently with kernel threads.
That could be a good solution for the issue. :)
If with CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED enabled, maybe we should set ksoftirqd’s cookie to be cookie-0 to solve the issue.
Thx.
Regards,
Jiang
Quoted 61 lines
Thanks,
-AubreyQuoted 55 lines
Then it will have cookie 0 and be able to
share core with kernel threads. About user-user isolation that you need, if you tag any "untrusted" threads by adding it to CGroup, then there will automatically isolated from uperf while allowing uperf to share CPU with kernel threads.Please let me know your thoughts and thanks,
- Joel
Quoted 45 lines
Thanks,
-AubreyQuoted 41 lines
5. Document everything clearly so the semantics are clear both to the developers of core scheduling and to system administrators.
Note that, with the concept of "system trusted cookie", we can also do optimizations like:
1. Disable STIBP when switching into trusted tasks.
2. Disable L1D flushing / verw stuff for L1TF/MDS issues, when switching into trusted tasks.At least #1 seems to be biting enabling HT on ChromeOS right now, and one other engineer requested I do something like #2 already.
Once we get full-syscall isolation working, threads belonging to a process can also share a core so those can just share a core with the task-group leader.
Quoted 4 lines
Quoted 1 line
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
This is a good question, from the data we measured by uperf, SMT+core-scheduling is 28.2% worse than no-SMT, :(
This is worrying for sure. :-(. We ought to debug/profile it more to see what is causing the overhead. Me/Vineeth added it as a topic for LPC as well.
Any other thoughts from others on this?
thanks,
- Joel
Quoted 9 lines
Quoted 8 lines
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Re: [RFC PATCH 00/16] Core scheduling v6(Internet mail)
Quoted 69 lines · benbjiang(蒋彪)
Quoted 69 lines · Aubrey
Quoted 63 lines · Joel Fernandes
Quoted 45 lines · Aubrey
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).Quoted 5 lines
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Yes, but let me try to understand better. There are 2 problems here I think:
1. ksoftirqd getting idled when HT is turned on, because uperf is sharing a core with it: This should not be any worse than SMT OFF, because even SMT OFF would also reduce ksoftirqd's CPU time just core sched is doing. Sure core-scheduling adds some overhead with IPIs but such a huge drop of perf is strange. Peter any thoughts on that?
2. Interface: To solve the performance problem, you are saying you want uperf to share a core with ksoftirqd so that it is not forced into idle. Why not just keep uperf out of the cgroup?
Quoted 7 lines
Quoted 1 line
I guess this is unacceptable for who runs their apps in container and vm.
IMHO, just as Joel proposed,
1. Consider all cookie-0 task as trusted.
2. All kernel threads and idle task would have a cookie 0
In that way, all tasks with cookies(including uperf in a cgroup) could run concurrently with kernel threads.
That could be a good solution for the issue. :)
From uperf point of review, it can trust cookie-0(I assume we still need some modifications to change cookie-match to cookie-compatible to allow ZERO and NONZERO run together).
But from kernel thread point of review, it can NOT trust uperf, unless we set uperf's cookie to 0.
Thanks,
-Aubrey
Re: [RFC PATCH 00/16] Core scheduling v6(Internet mail)
Quoted 82 lines · Aubrey
Quoted 69 lines · benbjiang(蒋彪)
Quoted 69 lines · Aubrey
Quoted 63 lines · Joel Fernandes
Quoted 45 lines · Aubrey
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).Quoted 5 lines
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Yes, but let me try to understand better. There are 2 problems here I think:
1. ksoftirqd getting idled when HT is turned on, because uperf is sharing a core with it: This should not be any worse than SMT OFF, because even SMT OFF would also reduce ksoftirqd's CPU time just core sched is doing. Sure core-scheduling adds some overhead with IPIs but such a huge drop of perf is strange. Peter any thoughts on that?
2. Interface: To solve the performance problem, you are saying you want uperf to share a core with ksoftirqd so that it is not forced into idle. Why not just keep uperf out of the cgroup?
Quoted 7 lines
Quoted 1 line
I guess this is unacceptable for who runs their apps in container and vm.
IMHO, just as Joel proposed,
1. Consider all cookie-0 task as trusted.
2. All kernel threads and idle task would have a cookie 0
In that way, all tasks with cookies(including uperf in a cgroup) could run concurrently with kernel threads.
That could be a good solution for the issue. :)
Quoted 6 lines
From uperf point of review, it can trust cookie-0(I assume we still need some modifications to change cookie-match to cookie-compatible to allow ZERO and NONZERO run together).
But from kernel thread point of review, it can NOT trust uperf, unless we set uperf's cookie to 0.
That’s right. :)
Could we set the cookie of cgroup where uperf lies to 0?
Thx.
Regards,
Jiang
Quoted 4 lines
Thanks,
-Aubrey
Re: [RFC PATCH 00/16] Core scheduling v6(Internet mail)
Quoted 82 lines · benbjiang(蒋彪)
Quoted 82 lines · Aubrey
Quoted 76 lines · benbjiang(蒋彪)
Quoted 65 lines · Aubrey
Quoted 60 lines · Joel Fernandes
Quoted 45 lines · Aubrey
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).Quoted 5 lines
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Yes, but let me try to understand better. There are 2 problems here I think:
1. ksoftirqd getting idled when HT is turned on, because uperf is sharing a core with it: This should not be any worse than SMT OFF, because even SMT OFF would also reduce ksoftirqd's CPU time just core sched is doing. Sure core-scheduling adds some overhead with IPIs but such a huge drop of perf is strange. Peter any thoughts on that?
Quoted 5 lines
Quoted 3 lines
2. Interface: To solve the performance problem, you are saying you want uperf to share a core with ksoftirqd so that it is not forced into idle. Why not just keep uperf out of the cgroup?
I guess this is unacceptable for who runs their apps in container and vm.
IMHO, just as Joel proposed,
1. Consider all cookie-0 task as trusted.
2. All kernel threads and idle task would have a cookie 0
In that way, all tasks with cookies(including uperf in a cgroup) could run concurrently with kernel threads.
That could be a good solution for the issue. :)
Quoted 9 lines
Quoted 6 lines
From uperf point of review, it can trust cookie-0(I assume we still need some modifications to change cookie-match to cookie-compatible to allow ZERO and NONZERO run together).
But from kernel thread point of review, it can NOT trust uperf, unless we set uperf's cookie to 0.
That’s right. :)
Could we set the cookie of cgroup where uperf lies to 0?
IMHO the disadvantage is that if there are two or more cgroups set cookie-0, then the user applications in these cgroups could run concurrently on a core, though all of them are set as trusted, we made a hole of user->user isolation.
Thanks,
-Aubrey
Re: [RFC PATCH 00/16] Core scheduling v6(Internet mail)
Hi,
Quoted 89 lines · Aubrey
Quoted 86 lines · benbjiang(蒋彪)
Quoted 86 lines · Aubrey
Quoted 69 lines · benbjiang(蒋彪)
Quoted 69 lines · Aubrey
Quoted 63 lines · Joel Fernandes
Quoted 45 lines · Aubrey
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).Quoted 5 lines
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Yes, but let me try to understand better. There are 2 problems here I think:
1. ksoftirqd getting idled when HT is turned on, because uperf is sharing a core with it: This should not be any worse than SMT OFF, because even SMT OFF would also reduce ksoftirqd's CPU time just core sched is doing. Sure core-scheduling adds some overhead with IPIs but such a huge drop of perf is strange. Peter any thoughts on that?
2. Interface: To solve the performance problem, you are saying you want uperf to share a core with ksoftirqd so that it is not forced into idle. Why not just keep uperf out of the cgroup?
Quoted 7 lines
Quoted 1 line
I guess this is unacceptable for who runs their apps in container and vm.
IMHO, just as Joel proposed,
1. Consider all cookie-0 task as trusted.
2. All kernel threads and idle task would have a cookie 0
In that way, all tasks with cookies(including uperf in a cgroup) could run concurrently with kernel threads.
That could be a good solution for the issue. :)From uperf point of review, it can trust cookie-0(I assume we still need some modifications to change cookie-match to cookie-compatible to allow ZERO and NONZERO run together).
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
But from kernel thread point of review, it can NOT trust uperf, unless we set uperf's cookie to 0.
That’s right. :)
Could we set the cookie of cgroup where uperf lies to 0?IMHO the disadvantage is that if there are two or more cgroups set cookie-0, then the user applications in these cgroups could run concurrently on a core, though all of them are set as trusted, we made a hole of
user->userisolation.
For that case, how about,
- use a special cookie(cookie-trust) instead of cookie-0 for kernel thread
- implement cookie_partial_match() to match part of the cookie
- Cookie-normal(normal tasks use) could trust cookie-trust,
- tasks tend to be trusted by cookie-trust could use cookies including cookie-trust partially, while cookie-normal does not include cookie-trust.
- cookie-trust tasks use cookie_partial_match() to match cookie
- normal tasks use the standard cookie match(full match) interface to match cookie.
Just a sudden thought. :)
Thx.
Regards,
Jiang
Quoted 61 lines · Aubrey
Quoted 60 lines · Joel Fernandes
Quoted 45 lines · Aubrey
Hi Joel,
Quoted 42 lines · Joel Fernandes
Hi Aubrey,
Apologies for replying late as I was still looking into the details.
On Wed, Aug 05, 2020 at 11:57:20AM +0800, Li, Aubrey wrote:
[...]Quoted 17 lines
+/*
+ * Core scheduling policy:
+ * -CORE_SCHED_DISABLED: core scheduling is disabled.
+ * -CORE_COOKIE_MATCH: tasks with same cookie can run
+ * on the same core concurrently.
+ * -CORE_COOKIE_TRUST: trusted task can run with kernel thread on the same core concurrently.
+ * -CORE_COOKIE_LONELY: tasks with cookie can run only
+ * with idle thread on the same core.
+ */(quoted diff) +6
-0+enum coresched_policy { + CORE_SCHED_DISABLED, + CORE_SCHED_COOKIE_MATCH, + CORE_SCHED_COOKIE_TRUST, + CORE_SCHED_COOKIE_LONELY, +};Quoted 2 lines
We can set policy to
CORE_COOKIE_TRUSTof uperf cgroup and fix this kind of performance regression. Not sure if this sounds attractive?Instead of this, I think it can be something simpler IMHO:
1. Consider all cookie-0 task as trusted. (Even right now, if you apply the core-scheduling patchset, such tasks will share a core and sniff on each other. So let us not pretend that such tasks are not trusted).
2. All kernel threads and idle task would have a cookie 0 (so that will cover ksoftirqd reported in your original issue).
3. Add a config option (
CONFIG_SCHED_CORE_DEFAULT_TASKS_UNTRUSTED). Default enable it. Setting this option would tag all tasks that are forked from a cookie-0 task with their own cookie. Later on, such tasks can be added to a group. This cover's PeterZ's ask about having 'default untrusted'). (Users like ChromeOS that don't want to userspace system processes to be tagged can disable this option so such tasks will be cookie-0).Quoted 5 lines
Quoted 2 lines
4. Allow prctl/cgroup interfaces to create groups of tasks and override the above behaviors.
How does uperf in a cgroup work with ksoftirqd? Are you suggesting I set uperf's cookie to be cookie-0 via prctl?
Yes, but let me try to understand better. There are 2 problems here I think:
1. ksoftirqd getting idled when HT is turned on, because uperf is sharing a core with it: This should not be any worse than SMT OFF, because even SMT OFF would also reduce ksoftirqd's CPU time just core sched is doing. Sure core-scheduling adds some overhead with IPIs but such a huge drop of perf is strange. Peter any thoughts on that?
Quoted 5 lines
Quoted 3 lines
2. Interface: To solve the performance problem, you are saying you want uperf to share a core with ksoftirqd so that it is not forced into idle. Why not just keep uperf out of the cgroup?
I guess this is unacceptable for who runs their apps in container and vm.
I think let us forget about #2, that's just a workaround. #1 is probably what we should look into for your problem. Was talking to Vineeth earlier, is it possible that the fairness issues that Aaron and Peter are looking into is causing the performance problem here?
So like, if ksoftirqd being higher prio is making the vruntime delta between 2 CFS tasks sharing a core to be quite high, then it causes the core-wide min_vruntime to be high. Then if uperf gets enqueued, it will get starved by ksoftirqd and not able to run till ksoftirqd's vruntime catches up.
Other than that, the only other thing (AFAIK) is the IPI/scheduler overhead is giving uperf worse performance than SMT-off and we ought to reduce the overhead some how. Does a kernel perf profile show you any smoking guns?
thanks,
- Joel
Quoted 60 lines
Thanks,
-AubreyQuoted 55 lines
Then it will have cookie 0 and be able to
share core with kernel threads. About user-user isolation that you need, if you tag any "untrusted" threads by adding it to CGroup, then there will automatically isolated from uperf while allowing uperf to share CPU with kernel threads.Please let me know your thoughts and thanks,
- Joel
Quoted 45 lines
Thanks,
-AubreyQuoted 41 lines
5. Document everything clearly so the semantics are clear both to the developers of core scheduling and to system administrators.
Note that, with the concept of "system trusted cookie", we can also do optimizations like:
1. Disable STIBP when switching into trusted tasks.
2. Disable L1D flushing / verw stuff for L1TF/MDS issues, when switching into trusted tasks.At least #1 seems to be biting enabling HT on ChromeOS right now, and one other engineer requested I do something like #2 already.
Once we get full-syscall isolation working, threads belonging to a process can also share a core so those can just share a core with the task-group leader.
Quoted 4 lines
Quoted 1 line
Is the uperf throughput worse with SMT+core-scheduling versus no-SMT ?
This is a good question, from the data we measured by uperf, SMT+core-scheduling is 28.2% worse than no-SMT, :(
This is worrying for sure. :-(. We ought to debug/profile it more to see what is causing the overhead. Me/Vineeth added it as a topic for LPC as well.
Any other thoughts from others on this?
thanks,
- Joel
Quoted 9 lines
Quoted 8 lines
thanks,
- Joel
PS: I am planning to write a patch behind a CONFIG option that tags all processes (default untrusted) so everything gets a cookie which some folks said was how they wanted (have a whitelist instead of blacklist).
Hi Vineeth,
Quoted 47 lines · Vineeth Remanan Pillai
Sixth iteration of the Core-Scheduling feature.
Core scheduling is a feature that allows only trusted tasks to run concurrently on cpus sharing compute resources (eg: hyperthreads on a core). The goal is to mitigate the core-level side-channel attacks without requiring to disable SMT (which has a significant impact on performance in some situations). Core scheduling (as of v6) mitigates user-space to user-space attacks and user to kernel attack when one of the siblings enters the kernel via interrupts. It is still possible to have a task attack the sibling thread when it enters the kernel via syscalls.
By default, the feature doesn't change any of the current scheduler behavior. The user decides which tasks can run simultaneously on the same core (for now by having them in the same tagged cgroup). When a tag is enabled in a cgroup and a task from that cgroup is running on a hardware thread, the scheduler ensures that only idle or trusted tasks run on the other sibling(s). Besides security concerns, this feature can also be beneficial for RT and performance applications where we want to control how tasks make use of SMT dynamically.
This iteration is mostly a cleanup of v5 except for a major feature of pausing sibling when a cpu enters kernel via nmi/irq/softirq. Also introducing documentation and includes minor crash fixes.
One major cleanup was removing the hotplug support and related code. The hotplug related crashes were not documented and the fixes piled up over time leading to complex code. We were not able to reproduce the crashes in the limited testing done. But if they are reroducable, we don't want to hide them. We should document them and design better fixes if any.
In terms of performance, the results in this release are similar to v5. On a x86 system with N hardware threads:
- if only N/2 hardware threads are busy, the performance is similar between baseline, corescheduling and nosmt
- if N hardware threads are busy with N different corescheduling groups, the impact of corescheduling is similar to nosmt
- if N hardware threads are busy and multiple active threads share the same corescheduling cookie, they gain a performance improvement over nosmt.
The specific performance impact depends on the workload, but for a really busy database 12-vcpu VM (1 coresched tag) running on a 36 hardware threads NUMA node with 96 mostly idle neighbor VMs (each in their own coresched tag), the performance drops by 54% with corescheduling and drops by 90% with nosmt.
Quoted 2 lines
v6 is rebased on 5.7.6(a06eb423367e)
https://github.com/digitalocean/linux-coresched/tree/coresched/v6-v5.7.y
As discussed during Linux Plumbers, here is a small repo with test scripts and applications that I've used to look at core scheduling unfairness:
https://github.com/agraf/schedgaps
Please let me know if it's unclear how to use it or if you see issues in your environment.
Please also make sure to only run this on idle server class hardware. Notebooks will most definitely have too many uncontrollable sources of timing entropy to give sensible results.
Alex
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Hi Alex,
Quoted 7 lines
As discussed during Linux Plumbers, here is a small repo with test scripts and applications that I've used to look at core scheduling unfairness:
Thanks for sharing :).
Quoted 3 lines
Please let me know if it's unclear how to use it or if you see issues in your environment.
Will give it a try soon and let you know. Went through the README quickly and documentation is very clear.
This is really helpful and would be really useful in future testing as well.
Thanks,
Vineeth