Fix gatekeeper affinity
PATCH 1/1 Fix gatekeeper affinity
As the xnsched structures get initialized later, during xnpod_init, xnsched_cpu always returned 0 in the gatekeeper_thread prologue. That caused binding of all gatekeepers to CPU 0.
---
ksrc/nucleus/shadow.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
ksrc/nucleus/shadow.c +4 -1
@@ -823,11 +823,14 @@ static int gatekeeper_thread(void *data)struct task_struct *this_task = current;DECLARE_WAITQUEUE (wait, this_task);struct xnsched *sched = data; -int cpu =xnsched_cpu (sched);struct xnthread *target;cpumask_t cpumask; +int cpu;spl_t s; +/* sched not fully initialized, xnsched_cpu does not work yet */ + cpu = sched - nkpod_struct.sched; + this_task->flags |=PF_NOFREEZE ;sigfillset (&this_task->blocked); cpumask =cpumask_of_cpu (cpu);
Quoted 20 lines · Jan Kiszka
As the xnsched structures get initialized later, during
xnpod_init,xnsched_cpualways returned 0 in thegatekeeper_threadprologue. That caused binding of all gatekeepers to CPU 0.Signed-off-by Jan Kiszka <[email protected]>---
ksrc/nucleus/shadow.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)ksrc/nucleus/shadow.c +0
-1@@ -823,11 +823,14 @@ static int gatekeeper_thread(void *data)struct task_struct *this_task = current;DECLARE_WAITQUEUE (wait, this_task);struct xnsched *sched = data; -int cpu =xnsched_cpu (sched);struct xnthread *target;
ksrc/nucleus/shadow.c static int gatekeeper_thread(void *data) · line 836
Quoted 10 lines
ksrc/nucleus/shadow.c +4
-0cpumask_t cpumask; +int cpu;spl_t s; +/* sched not fully initialized, xnsched_cpu does not work yet */ + cpu = sched - nkpod_struct.sched; + this_task->flags |=PF_NOFREEZE ;sigfillset (&this_task->blocked); cpumask =cpumask_of_cpu (cpu);
This does not look good, it means that the gatekeeper accesses the sched structure before it is initialized. So, IMO, the proper fix would be to start the gatekeepers only after the sched structure has been initialized.
Signature · 1 line
Gilles.
Quoted 25 lines · Gilles Chanteperdrix
Quoted 24 lines · Jan Kiszka
As the xnsched structures get initialized later, during
xnpod_init,xnsched_cpualways returned 0 in thegatekeeper_threadprologue. That caused binding of all gatekeepers to CPU 0.Signed-off-by Jan Kiszka <[email protected]>---
ksrc/nucleus/shadow.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)ksrc/nucleus/shadow.c +1
-1@@ -823,11 +823,14 @@ static int gatekeeper_thread(void *data)struct task_struct *this_task = current;DECLARE_WAITQUEUE (wait, this_task);struct xnsched *sched = data; -int cpu =xnsched_cpu (sched);struct xnthread *target;cpumask_t cpumask; +int cpu;spl_t s;
Quoted 11 lines
ksrc/nucleus/shadow.c static int gatekeeper_thread(void *data) · line 836
Quoted 6 lines
ksrc/nucleus/shadow.c +3
-0+/* sched not fully initialized, xnsched_cpu does not work yet */ + cpu = sched - nkpod_struct.sched; + this_task->flags |=PF_NOFREEZE ;sigfillset (&this_task->blocked); cpumask =cpumask_of_cpu (cpu);This does not look good, it means that the gatekeeper accesses the sched structure before it is initialized. So, IMO, the proper fix would be to start the gatekeepers only after the sched structure has been initialized.
I briefly thought about moving xnshadow_mount into xnpod_init. But given the fact that it worked like this before and that I was not able to quickly exclude new regressions when reordering things, I decided to restore the old pattern.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2 Corporate Competence Center Embedded Linux
Quoted 37 lines · Jan Kiszka
Quoted 25 lines · Gilles Chanteperdrix
Quoted 24 lines · Jan Kiszka
As the xnsched structures get initialized later, during
xnpod_init,xnsched_cpualways returned 0 in thegatekeeper_threadprologue. That caused binding of all gatekeepers to CPU 0.Signed-off-by Jan Kiszka <[email protected]>---
ksrc/nucleus/shadow.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)ksrc/nucleus/shadow.c +1
-1@@ -823,11 +823,14 @@ static int gatekeeper_thread(void *data)struct task_struct *this_task = current;DECLARE_WAITQUEUE (wait, this_task);struct xnsched *sched = data; -int cpu =xnsched_cpu (sched);struct xnthread *target;cpumask_t cpumask; +int cpu;spl_t s;Quoted 10 lines
ksrc/nucleus/shadow.c static int gatekeeper_thread(void *data) · line 836
Quoted 6 lines
ksrc/nucleus/shadow.c +3
-0+/* sched not fully initialized, xnsched_cpu does not work yet */ + cpu = sched - nkpod_struct.sched; + this_task->flags |=PF_NOFREEZE ;sigfillset (&this_task->blocked); cpumask =cpumask_of_cpu (cpu);This does not look good, it means that the gatekeeper accesses the sched structure before it is initialized. So, IMO, the proper fix would be to start the gatekeepers only after the sched structure has been initialized.
Quoted 4 lines
I briefly thought about moving
xnshadow_mountintoxnpod_init. But given the fact that it worked like this before and that I was not able to quickly exclude new regressions when reordering things, I decided to restore the old pattern.
Ok. What about passing cpu as the gatekeeper argument, and using xnpod_sched_slot(cpu) to find the sched pointer ?
Signature · 1 line
Gilles.
Quoted 37 lines · Gilles Chanteperdrix
Quoted 36 lines · Jan Kiszka
Quoted 35 lines · Gilles Chanteperdrix
Quoted 20 lines · Jan Kiszka
As the xnsched structures get initialized later, during
xnpod_init,xnsched_cpualways returned 0 in thegatekeeper_threadprologue. That caused binding of all gatekeepers to CPU 0.Signed-off-by Jan Kiszka <[email protected]>---
ksrc/nucleus/shadow.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)ksrc/nucleus/shadow.c +0
-1@@ -823,11 +823,14 @@ static int gatekeeper_thread(void *data)struct task_struct *this_task = current;DECLARE_WAITQUEUE (wait, this_task);struct xnsched *sched = data; -int cpu =xnsched_cpu (sched);struct xnthread *target;ksrc/nucleus/shadow.c static int gatekeeper_thread(void *data) · line 836
Quoted 10 lines
ksrc/nucleus/shadow.c +4
-0cpumask_t cpumask; +int cpu;spl_t s; +/* sched not fully initialized, xnsched_cpu does not work yet */ + cpu = sched - nkpod_struct.sched; + this_task->flags |=PF_NOFREEZE ;sigfillset (&this_task->blocked); cpumask =cpumask_of_cpu (cpu);This does not look good, it means that the gatekeeper accesses the sched structure before it is initialized. So, IMO, the proper fix would be to start the gatekeepers only after the sched structure has been initialized.
Quoted 8 lines
Quoted 4 lines
I briefly thought about moving
xnshadow_mountintoxnpod_init. But given the fact that it worked like this before and that I was not able to quickly exclude new regressions when reordering things, I decided to restore the old pattern.Ok. What about passing cpu as the gatekeeper argument, and using
xnpod_sched_slot(cpu)to find the sched pointer ?
Something like this? Same result, but looks indeed a bit nicer.
----------->
As the xnsched structures get initialized later, during xnpod_init, xnsched_cpu always returned 0 in the gatekeeper_thread prologue. That caused binding of all gatekeepers to CPU 0.
---
ksrc/nucleus/shadow.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
ksrc/nucleus/shadow.c +4 -4
@@ -822,8 +822,8 @@ static int gatekeeper_thread(void *data) {struct task_struct *this_task = current;DECLARE_WAITQUEUE (wait, this_task); -struct xnsched *sched = data; -int cpu =xnsched_cpu (sched); +int cpu = (long )data; +struct xnsched *sched =xnpod_sched_slot (cpu);struct xnthread *target;cpumask_t cpumask;spl_t s;
@@ -2600,8 +2600,8 @@ int xnshadow_mount(void)sema_init (&sched->gksync,0 );xnarch_memory_barrier (); sched->gatekeeper = -kthread_create (&gatekeeper_thread, sched,"gatekeeper/%d" , - cpu); +kthread_create (&gatekeeper_thread, (void *)(long )cpu, +"gatekeeper/%d" , cpu);wake_up_process (sched->gatekeeper);down (&sched->gksync); }
Quoted 25 lines · Gilles Chanteperdrix
Quoted 24 lines · Jan Kiszka
As the xnsched structures get initialized later, during
xnpod_init,xnsched_cpualways returned 0 in thegatekeeper_threadprologue. That caused binding of all gatekeepers to CPU 0.Signed-off-by Jan Kiszka <[email protected]>---
ksrc/nucleus/shadow.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)ksrc/nucleus/shadow.c +1
-1@@ -823,11 +823,14 @@ static int gatekeeper_thread(void *data)struct task_struct *this_task = current;DECLARE_WAITQUEUE (wait, this_task);struct xnsched *sched = data; -int cpu =xnsched_cpu (sched);struct xnthread *target;cpumask_t cpumask; +int cpu;spl_t s;
Quoted 9 lines
ksrc/nucleus/shadow.c static int gatekeeper_thread(void *data) · line 836
Quoted 6 lines
ksrc/nucleus/shadow.c +3
-0+/* sched not fully initialized, xnsched_cpu does not work yet */ + cpu = sched - nkpod_struct.sched; + this_task->flags |=PF_NOFREEZE ;sigfillset (&this_task->blocked); cpumask =cpumask_of_cpu (cpu);This does not look good, it means that the gatekeeper accesses the sched structure before it is initialized.
Fortunately, no, this can't be. The gatekeeper only refers to the semaphore and the sync barrier and only that. Since the semaphore is initialized when the gk starts, the gk sets up the barrier on entry, and the barrier can't be signaled until the nucleus has fully initialized in xnpod_init(), we used to be safe.
So, IMO, the proper fix would be to
Quoted 2 lines
start the gatekeepers only after the sched structure has been initialized.
Signature · 1 line
Philippe.
Quoted 4 lines · Jan Kiszka
As the xnsched structures get initialized later, during
xnpod_init,xnsched_cpualways returned 0 in thegatekeeper_threadprologue. That caused binding of all gatekeepers to CPU 0.
Hey, nice. This is why v2.4.x used pointer arithmetics to determine the cpu # in the first place.
Quoted 32 lines
Signed-off-by Jan Kiszka <[email protected]>---
ksrc/nucleus/shadow.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)ksrc/nucleus/shadow.c +4
-1@@ -823,11 +823,14 @@ static int gatekeeper_thread(void *data)struct task_struct *this_task = current;DECLARE_WAITQUEUE (wait, this_task);struct xnsched *sched = data; -int cpu =xnsched_cpu (sched);struct xnthread *target;cpumask_t cpumask; +int cpu;spl_t s; +/* sched not fully initialized, xnsched_cpu does not work yet */ + cpu = sched - nkpod_struct.sched; + this_task->flags |=PF_NOFREEZE ;sigfillset (&this_task->blocked); cpumask =cpumask_of_cpu (cpu);_______________________________________________
Xenomai-core mailing list
[email protected]
https://mail.gna.org/listinfo/xenomai-core
Signature · 1 line
Philippe.