perf/x86/amd/uncore: fixup use after free due to hotplug conversion
Subsystems (2)
PATCH 1/1 perf/x86/amd/uncore: fixup use after free due to hotplug conversion
The conversion change the behaviour of how the hotplug hooks were
invoked: Before one CPU at a time invoked them all. Now on registration each hook is invoked by all CPUs (one after the other) before the next state is invoked.
In amd_uncore_cpu_up_prepare() we allocate an uncore struct for each CPU with id 0. In amd_uncore_cpu_starting() we set the ID and search of others to replace it if available. CPU0 finds another struct with id 0 and puts its on the kill list. CPU1 find the struct of CPU0 and puts its on the kill list.
The online event will free them.
To fix this I use an id of -1 assuming that no uncore with such an id will show up. Step two is use a list to free the not needed elements.
--- arch/x86/events/amd/uncore.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
arch/x86/events/amd/uncore.c +18 -4
@@ -29,6 +29,8 @@#define COUNTER_SHIFT 16 +static HLIST_HEAD (uncore_unused_list); +struct amd_uncore {int id;int refcnt;
@@ -39,7 +41,7 @@ struct amd_uncore {cpumask_t *active_mask;struct pmu *pmu;struct perf_event *events[MAX_COUNTERS ]; -struct amd_uncore *free_when_cpu_online; +struct hlist_node node; };static struct amd_uncore *__percpu *amd_uncore_nb;
@@ -306,6 +308,7 @@ static int amd_uncore_cpu_up_prepare(unsigned int cpu) uncore_nb->msr_base =MSR_F15H_NB_PERF_CTL ; uncore_nb->active_mask = &amd_nb_active_mask; uncore_nb->pmu = &amd_nb_pmu; + uncore_nb->id =-1 ; *per_cpu_ptr (amd_uncore_nb, cpu) = uncore_nb; }
@@ -319,6 +322,7 @@ static int amd_uncore_cpu_up_prepare(unsigned int cpu) uncore_l2->msr_base =MSR_F16H_L2I_PERF_CTL ; uncore_l2->active_mask = &amd_l2_active_mask; uncore_l2->pmu = &amd_l2_pmu; + uncore_l2->id =-1 ; *per_cpu_ptr (amd_uncore_l2, cpu) = uncore_l2; }
@@ -348,7 +352,7 @@ amd_uncore_find_online_sibling(struct amd_uncore *this,continue ;if (this->id == that->id) { - that->free_when_cpu_online = this; +hlist_add_head (&this->node, &uncore_unused_list); this = that;break ; }
@@ -388,13 +392,23 @@ static int amd_uncore_cpu_starting(unsigned int cpu)return 0 ; } +static void uncore_clean_online (void ) +{ +struct amd_uncore *uncore; +struct hlist_node *n; + +hlist_for_each_entry_safe (uncore, n, &uncore_unused_list, node) { +hlist_del (&uncore->node); +kfree (uncore); + } +} +static void uncore_online (unsigned int cpu,struct amd_uncore *__percpu *uncores) {struct amd_uncore *uncore = *per_cpu_ptr (uncores, cpu); -kfree (uncore->free_when_cpu_online); - uncore->free_when_cpu_online =NULL ; +uncore_clean_online ();if (cpu == uncore->cpu)cpumask_set_cpu (cpu, uncore->active_mask);
Signature · 1 line
2.9.3
tip-bot for Sebastian Andrzej Siewior
merge bot[tip:perf/urgent] perf/x86/amd/uncore: Prevent use after free
perf/x86/amd/uncore: Prevent use after free
Commit-ID: 7d762e49c2117d3829eb3355f2617aea080ed3a7 Gitweb: http://git.kernel.org/tip/7d762e49c2117d3829eb3355f2617aea080ed3a7 Author: Sebastian Andrzej Siewior <[email protected]> AuthorDate: Fri, 9 Sep 2016 18:08:23 +0200 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 10 Sep 2016 00:00:06 +0200
perf/x86/amd/uncore: Prevent use after free
The resent conversion of the cpu hotplug support in the uncore driver introduced a regression due to the way the callbacks are invoked at initialization time.
The old code called the prepare/starting/online function on each online cpu as a block. The new code registers the hotplug callbacks in the core for each state. The core invokes the callbacks at each registration on all online cpus.
The code implicitely relied on the prepare/starting/online callbacks being called as combo on a particular cpu, which was not obvious and completely undocumented.
The resulting subtle wreckage happens due to the way how the uncore code manages shared data structures for cpus which share an uncore resource in hardware. The sharing is determined in the cpu starting callback, but the prepare callback allocates per cpu data for the upcoming cpu because potential sharing is unknown at this point. If the starting callback finds a online cpu which shares the hardware resource it takes a refcount on the percpu data of that cpu and puts the own data structure into a 'free_at_online' pointer of that shared data structure. The online callback frees that.
With the old model this worked because in a starting callback only one non unused structure (the one of the starting cpu) was available. The new code allocates the data structures for all cpus when the prepare callback is registered.
Now the starting function iterates through all online cpus and looks for a data structure (skipping its own) which has a matching hardware id. The id member of the data structure is initialized to 0, but the hardware id can be 0 as well. The resulting wreckage is:
CPU0 finds a matching id on CPU1, takes a refcount on CPU1 data and puts its own data structure into CPU1s data structure to be freed.
CPU1 skips CPU0 because the data structure is its allegedly unsued own. It finds a matching id on CPU2, takes a refcount on CPU1 data and puts its own data structure into CPU2s data structure to be freed.
....
Now the online callbacks are invoked.
CPU0 has a pointer to CPU1s data and frees the original CPU0 data. So far so good.
CPU1 has a pointer to CPU2s data and frees the original CPU1 data, which is still referenced by CPU0 ---> Booom
So there are two issues to be solved here:
1) The id field must be initialized at allocation time to a value which cannot be a valid hardware id, i.e. -1
This prevents the above scenario, but now CPU1 and CPU2 both stick their own data structure into the free_at_online pointer of CPU0. So we leak CPU1s data structure.
2) Fix the memory leak described in #1
Instead of having a single pointer, use a hlist to enqueue the superflous data structures which are then freed by the first cpu invoking the online callback.
Ideally we should know the sharing _before_ invoking the prepare callback, but that's way beyond the scope of this bug fix.
[ tglx: Rewrote changelog ]
--- arch/x86/events/amd/uncore.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
arch/x86/events/amd/uncore.c +18 -4
@@ -29,6 +29,8 @@#define COUNTER_SHIFT 16 +static HLIST_HEAD (uncore_unused_list); +struct amd_uncore {int id;int refcnt;
@@ -39,7 +41,7 @@ struct amd_uncore {cpumask_t *active_mask;struct pmu *pmu;struct perf_event *events[MAX_COUNTERS ]; -struct amd_uncore *free_when_cpu_online; +struct hlist_node node; };static struct amd_uncore *__percpu *amd_uncore_nb;
@@ -306,6 +308,7 @@ static int amd_uncore_cpu_up_prepare(unsigned int cpu) uncore_nb->msr_base =MSR_F15H_NB_PERF_CTL ; uncore_nb->active_mask = &amd_nb_active_mask; uncore_nb->pmu = &amd_nb_pmu; + uncore_nb->id =-1 ; *per_cpu_ptr (amd_uncore_nb, cpu) = uncore_nb; }
@@ -319,6 +322,7 @@ static int amd_uncore_cpu_up_prepare(unsigned int cpu) uncore_l2->msr_base =MSR_F16H_L2I_PERF_CTL ; uncore_l2->active_mask = &amd_l2_active_mask; uncore_l2->pmu = &amd_l2_pmu; + uncore_l2->id =-1 ; *per_cpu_ptr (amd_uncore_l2, cpu) = uncore_l2; }
@@ -348,7 +352,7 @@ amd_uncore_find_online_sibling(struct amd_uncore *this,continue ;if (this->id == that->id) { - that->free_when_cpu_online = this; +hlist_add_head (&this->node, &uncore_unused_list); this = that;break ; }
@@ -388,13 +392,23 @@ static int amd_uncore_cpu_starting(unsigned int cpu)return 0 ; } +static void uncore_clean_online (void ) +{ +struct amd_uncore *uncore; +struct hlist_node *n; + +hlist_for_each_entry_safe (uncore, n, &uncore_unused_list, node) { +hlist_del (&uncore->node); +kfree (uncore); + } +} +static void uncore_online (unsigned int cpu,struct amd_uncore *__percpu *uncores) {struct amd_uncore *uncore = *per_cpu_ptr (uncores, cpu); -kfree (uncore->free_when_cpu_online); - uncore->free_when_cpu_online =NULL ; +uncore_clean_online ();if (cpu == uncore->cpu)cpumask_set_cpu (cpu, uncore->active_mask);