Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4406CC432C0 for ; Fri, 29 Nov 2019 15:45:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 055D12075C for ; Fri, 29 Nov 2019 15:45:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=axtens.net header.i=@axtens.net header.b="eRxKLAUk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 055D12075C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=axtens.net Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 938D86B05A8; Fri, 29 Nov 2019 10:45:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C2256B05A9; Fri, 29 Nov 2019 10:45:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 789A56B05AA; Fri, 29 Nov 2019 10:45:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0097.hostedemail.com [216.40.44.97]) by kanga.kvack.org (Postfix) with ESMTP id 5DEE16B05A8 for ; Fri, 29 Nov 2019 10:45:28 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 20CBE180AD811 for ; Fri, 29 Nov 2019 15:45:28 +0000 (UTC) X-FDA: 76209739536.11.lock97_2fa0e91b69817 X-HE-Tag: lock97_2fa0e91b69817 X-Filterd-Recvd-Size: 8868 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 Nov 2019 15:45:27 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id o9so13026231plk.6 for ; Fri, 29 Nov 2019 07:45:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axtens.net; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=vsKvBmcEHkBcKrbZ7FMQtODXsXcFjKXnby3n743Jem8=; b=eRxKLAUkQ7j0Crv9q/2f0pvf4D4ZeeIFIFgK9WXy7RBTxfbRWQAZBKqifEEv6c9b9/ OIrFQy+q6JbAX6Byz9AeHc/gopoSqZRzKkd3hAKj7k0+bKg9n6mDzsJFZu67N+GTd8BI +Fb4/RI78bHBGxV0B0dXbSRQm1/9KX6McwbUM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=vsKvBmcEHkBcKrbZ7FMQtODXsXcFjKXnby3n743Jem8=; b=FKstZWRsQJ/7pi7SXU5+NrJMGKT+ZyLHYxJ09FMZ/Mlev6ogPaWCx+uTUY7bvJLfan IC2pyw//9s2WHiM4WtP86bqo9nUtOCzJKeFye+A99m4YlwTjtMsFOaX/0pAneyoM3yne +zvXbdoUBk+lzTEqf3E1S0o9BDye8FzKDlSIuMFmMTND37tgf9FAIsFZL/kS/2IPFZRl IPKu51RTTuPSlHfv8AuEU+fwlBD9CF41m2Yb4fm17nuSxuhrrS7dnpIRzgbXk2GyTz+e FRdwjdN46+Gu1Y88ZrR+nygAyviqPfyzNsgTYwZckS2UcwEsSgKt4672Bh21RcbjkbzY XR2Q== X-Gm-Message-State: APjAAAWJg7B4AGupS11Ne+cc+wkfC6l/yG3pEVD95K/Vy8g2y/qwBV7x 4hIMEVebEVXrku0Okt2SOHwBXw== X-Google-Smtp-Source: APXvYqw8MSfC4E3Jbalr12WOFgc6cezeghoctIkWKDOOAUhT/lf+3CB5ycpG307uSv1p1wytWWeNBw== X-Received: by 2002:a17:902:968b:: with SMTP id n11mr8143652plp.120.1575042325924; Fri, 29 Nov 2019 07:45:25 -0800 (PST) Received: from localhost (2001-44b8-111e-5c00-4092-39f5-bb9d-b59a.static.ipv6.internode.on.net. [2001:44b8:111e:5c00:4092:39f5:bb9d:b59a]) by smtp.gmail.com with ESMTPSA id 186sm25273018pfe.141.2019.11.29.07.45.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Nov 2019 07:45:25 -0800 (PST) From: Daniel Axtens To: kasan-dev@googlegroups.com, linux-mm@kvack.org, x86@kernel.org, aryabinin@virtuozzo.com, glider@google.com, linux-kernel@vger.kernel.org, dvyukov@google.com Cc: Daniel Axtens , Qian Cai Subject: [PATCH] kasan: support vmalloc backing of vm_map_ram() Date: Sat, 30 Nov 2019 02:45:19 +1100 Message-Id: <20191129154519.30964-1-dja@axtens.net> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This fixes some crashes in xfs, binder and the i915 mock_selftests, with kasan vmalloc, where no shadow space was being allocated when vm_map_ram was called. vm_map_ram has two paths, a path that uses vmap_block and a path that uses alloc_vmap_area. The alloc_vmap_area path is straight-forward, we handle it like most other allocations. For the vmap_block case, we map a shadow for the entire vmap_block when the block is allocated, and unpoison it piecewise in vm_map_ram(). It already gets cleaned up when the block is released in the lazy vmap area freeing path. For both cases, we need to tweak the interface to allow for vmalloc addresses that don't have an attached vm_struct. Reported-by: Dmitry Vyukov Cc: Qian Cai Thanks-to: Andrey Ryabinin Signed-off-by: Daniel Axtens --- include/linux/kasan.h | 6 ++++++ mm/kasan/common.c | 37 +++++++++++++++++++++++-------------- mm/vmalloc.c | 24 ++++++++++++++++++++++++ 3 files changed, 53 insertions(+), 14 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 4f404c565db1..0b50b59a8ff5 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -207,6 +207,7 @@ static inline void *kasan_reset_tag(const void *addr) #ifdef CONFIG_KASAN_VMALLOC int kasan_populate_vmalloc(unsigned long requested_size, struct vm_struct *area); +int kasan_populate_vmalloc_area(unsigned long size, void *addr); void kasan_poison_vmalloc(void *start, unsigned long size); void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, @@ -218,6 +219,11 @@ static inline int kasan_populate_vmalloc(unsigned lo= ng requested_size, return 0; } =20 +static inline int kasan_populate_vmalloc_area(unsigned long size, void *= addr) +{ + return 0; +} + static inline void kasan_poison_vmalloc(void *start, unsigned long size)= {} static inline void kasan_release_vmalloc(unsigned long start, unsigned long end, diff --git a/mm/kasan/common.c b/mm/kasan/common.c index df3371d5c572..27d8522ffaad 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -779,27 +779,15 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, = unsigned long addr, =20 int kasan_populate_vmalloc(unsigned long requested_size, struct vm_struc= t *area) { - unsigned long shadow_start, shadow_end; int ret; - - shadow_start =3D (unsigned long)kasan_mem_to_shadow(area->addr); - shadow_start =3D ALIGN_DOWN(shadow_start, PAGE_SIZE); - shadow_end =3D (unsigned long)kasan_mem_to_shadow(area->addr + - area->size); - shadow_end =3D ALIGN(shadow_end, PAGE_SIZE); - - ret =3D apply_to_page_range(&init_mm, shadow_start, - shadow_end - shadow_start, - kasan_populate_vmalloc_pte, NULL); + ret =3D kasan_populate_vmalloc_area(area->size, area->addr); if (ret) return ret; =20 - flush_cache_vmap(shadow_start, shadow_end); + area->flags |=3D VM_KASAN; =20 kasan_unpoison_shadow(area->addr, requested_size); =20 - area->flags |=3D VM_KASAN; - /* * We need to be careful about inter-cpu effects here. Consider: * @@ -838,6 +826,27 @@ int kasan_populate_vmalloc(unsigned long requested_s= ize, struct vm_struct *area) return 0; } =20 +int kasan_populate_vmalloc_area(unsigned long size, void *addr) +{ + unsigned long shadow_start, shadow_end; + int ret; + + shadow_start =3D (unsigned long)kasan_mem_to_shadow(addr); + shadow_start =3D ALIGN_DOWN(shadow_start, PAGE_SIZE); + shadow_end =3D (unsigned long)kasan_mem_to_shadow(addr + size); + shadow_end =3D ALIGN(shadow_end, PAGE_SIZE); + + ret =3D apply_to_page_range(&init_mm, shadow_start, + shadow_end - shadow_start, + kasan_populate_vmalloc_pte, NULL); + if (ret) + return ret; + + flush_cache_vmap(shadow_start, shadow_end); + + return 0; +} + /* * Poison the shadow for a vmalloc region. Called as part of the * freeing process at the time the region is freed. diff --git a/mm/vmalloc.c b/mm/vmalloc.c index bf030516258c..2896189e351f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1509,6 +1509,13 @@ static void *new_vmap_block(unsigned int order, gf= p_t gfp_mask) return ERR_CAST(va); } =20 + err =3D kasan_populate_vmalloc_area(VMAP_BLOCK_SIZE, va->va_start); + if (unlikely(err)) { + kfree(vb); + free_vmap_area(va); + return ERR_PTR(err); + } + err =3D radix_tree_preload(gfp_mask); if (unlikely(err)) { kfree(vb); @@ -1554,6 +1561,7 @@ static void free_vmap_block(struct vmap_block *vb) spin_unlock(&vmap_block_tree_lock); BUG_ON(tmp !=3D vb); =20 + /* free_vmap_area will take care of freeing the shadow */ free_vmap_area_noflush(vb->va); kfree_rcu(vb, rcu_head); } @@ -1780,6 +1788,8 @@ void vm_unmap_ram(const void *mem, unsigned int cou= nt) if (likely(count <=3D VMAP_MAX_ALLOC)) { debug_check_no_locks_freed(mem, size); vb_free(mem, size); + kasan_poison_vmalloc(mem, size); + return; } =20 @@ -1787,6 +1797,7 @@ void vm_unmap_ram(const void *mem, unsigned int cou= nt) BUG_ON(!va); debug_check_no_locks_freed((void *)va->va_start, (va->va_end - va->va_start)); + /* vmap area purging will clean up the KASAN shadow later */ free_unmap_vmap_area(va); } EXPORT_SYMBOL(vm_unmap_ram); @@ -1817,6 +1828,11 @@ void *vm_map_ram(struct page **pages, unsigned int= count, int node, pgprot_t pro if (IS_ERR(mem)) return NULL; addr =3D (unsigned long)mem; + + /* + * We don't need to call kasan_populate_vmalloc_area here, as + * it's done at block allocation time. + */ } else { struct vmap_area *va; va =3D alloc_vmap_area(size, PAGE_SIZE, @@ -1826,7 +1842,15 @@ void *vm_map_ram(struct page **pages, unsigned int= count, int node, pgprot_t pro =20 addr =3D va->va_start; mem =3D (void *)addr; + + if (kasan_populate_vmalloc_area(size, mem)) { + vm_unmap_ram(mem, count); + return NULL; + } } + + kasan_unpoison_shadow(mem, size); + if (vmap_page_range(addr, addr + size, prot, pages) < 0) { vm_unmap_ram(mem, count); return NULL; --=20 2.20.1