Skip to contentSkip to conversation

New: kernel BUG at mm/usercopy.c:99 from stress-ng procfs

Bug 209919 Andrew Morton 2 participants 3 messages Oct 28, 2020 – Oct 29, 2020 1 message not in the archive Memory management

lore
Share & tools
lore

Andrew Morton

author maintainer
#1 lore raw
In reply to a message not in the archive

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 18 lines · On Wed, 28 Oct 2020 15:49:15 +0000 [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=209919

            Bug ID: 209919
           Summary: kernel BUG at mm/usercopy.c:99 from stress-ng procfs
           Product: Memory Management
           Version: 2.5
    Kernel Version: 5.10.0-rc1
          Hardware: All
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: No
Quoted 2 lines

The procfs stressor from the stress-ng project triggers a kernel BUG in the 5.10.0-rc1 kernel on multiple architectures.

Thanks. A question from Kees, below...

Quoted 37 lines

x86_64:

[root@localhost stress-ng]# ./stress-ng --procfs 0 --timeout 60
stress-ng: info: [3031466] dispatching hogs: 4 procfs

[  974.088011] ICMPv6: process `stress-ng-procf' is using deprecated sysctl
(syscall) net.ipv6.neigh.enp0s29u1u1u5.retrans_time - use
net.ipv6.neigh.enp0s29u1u1u5.retrans_time_ms instead 
[  984.137351] usercopy: Kernel memory exposure attempt detected from SLUB
object 'kmalloc-128' (offset 127, size 3)! 
[  984.148917] ------------[ cut here ]------------ 
[  984.154089] kernel BUG at mm/usercopy.c:99! 
[  984.158813] invalid opcode: 0000 [#1] SMP PTI 
[  984.163771] CPU: 0 PID: 3031471 Comm: stress-ng-procf Tainted: G          I 
     5.10.0-rc1 #1 
[  984.173483] Hardware name: IBM IBM System X3250 M4 -[2583AC1]-/00D3729, BIOS
-[JQE158AUS-1.05]- 07/23/2013 
[  984.184260] RIP: 0010:usercopy_abort+0x74/0x76 
[  984.189219] Code: 67 5c 8b 51 48 0f 45 d6 49 c7 c3 73 f7 5f 8b 4c 89 d1 57
48 c7 c6 68 57 5e 8b 48 c7 c7 38 f8 5f 8b 49 0f 45 f3 e8 13 71 ff ff <0f> 0b 4c
89 e1 49 89 d8 44 89 ea 31 f6 48 29 c1 48 c7 c7 b5 f7 5f 
[  984.210177] RSP: 0018:ffff9c1f007b3dc0 EFLAGS: 00010286 
[  984.216000] RAX: 0000000000000066 RBX: 0000000000000003 RCX:
0000000000000000 
[  984.223965] RDX: ffff911f37c27e20 RSI: ffff911f37c19050 RDI:
ffff911f37c19050 
[  984.231929] RBP: ffff911e04cd1f82 R08: 0000000000000000 R09:
0000000000000000 
[  984.239893] R10: ffff9c1f007b3bf8 R11: ffffffff8bd711a8 R12:
ffff911e04cd1f7f 
[  984.247857] R13: 0000000000000001 R14: 0000000000000003 R15:
ffff911e009b19c0 
[  984.255821] FS:  00007fbabb42b180(0000) GS:ffff911f37c00000(0000)
knlGS:0000000000000000 
[  984.264915] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[  984.271520] CR2: 00007fbabb7ef000 CR3: 000000014e296001 CR4:
00000000001706f0 
[  984.279683] Call Trace:
Quoted 8 lines
[  984.282581]  __check_heap_object+0xe0/0x110 
[  984.287405]  __check_object_size+0x136/0x150 
[  984.292347]  proc_sys_call_handler+0x167/0x250 
[  984.297565]  new_sync_read+0x108/0x180 
[  984.302082]  vfs_read+0x174/0x1d0 
[  984.306126]  ksys_read+0x58/0xd0 
[  984.310022]  do_syscall_64+0x33/0x40 
[  984.314277]  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Can we determine which /proc/sys entries these are?

Quoted 117 lines
[  984.320201] RIP: 0033:0x7fbabb6099ac 
[  984.324514] Code: ec 28 48 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 89 fc
ff ff 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 31 c0 0f 05 <48> 3d 00
f0 ff ff 77 34 44 89 c7 48 89 44 24 08 e8 bf fc ff ff 48 
[  984.346368] RSP: 002b:00007fff47397340 EFLAGS: 00000246 ORIG_RAX:
0000000000000000 
[  984.355402] RAX: ffffffffffffffda RBX: 0000000000000000 RCX:
00007fbabb6099ac 
[  984.363971] RDX: 0000000000000060 RSI: 00007fff47397390 RDI:
0000000000000006 
[  984.372583] RBP: 0000000000000006 R08: 0000000000000000 R09:
0000000000000000 
[  984.381093] R10: 00000000000fa2b4 R11: 0000000000000246 R12:
0000000000000003 
[  984.389577] R13: 00007fff473a3630 R14: 0000000000001000 R15:
0000000000000060 
[  984.398087] Modules linked in: binfmt_misc rfkill sunrpc intel_rapl_msr

intel_rapl_common x86_pkg_temp_thermal mgag200 intel_powerclamp iTCO_wdt i2c_algo_bit coretemp intel_pmc_bxt cdc_ether gpio_ich drm_kms_helper iTCO_vendor_support usbnet mii cec rapl ipmi_ssif i2c_i801 intel_cstate e1000e intel_uncore ie31200_edac pcspkr ipmi_si i2c_smbus lpc_ich ipmi_devintf ipmi_msghandler drm ip_tables xfs crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_generic pata_acpi wmi

[  984.449316] ---[ end trace d44739bb135b1e63 ]--- 
[  984.455360] RIP: 0010:usercopy_abort+0x74/0x76 
[  984.461181] Code: 67 5c 8b 51 48 0f 45 d6 49 c7 c3 73 f7 5f 8b 4c 89 d1 57
48 c7 c6 68 57 5e 8b 48 c7 c7 38 f8 5f 8b 49 0f 45 f3 e8 13 71 ff ff <0f> 0b 4c
89 e1 49 89 d8 44 89 ea 31 f6 48 29 c1 48 c7 c7 b5 f7 5f 
[  984.483379] RSP: 0018:ffff9c1f007b3dc0 EFLAGS: 00010286 
[  984.489416] RAX: 0000000000000066 RBX: 0000000000000003 RCX:
0000000000000000 
[  984.497965] RDX: ffff911f37c27e20 RSI: ffff911f37c19050 RDI:
ffff911f37c19050 
[  984.507102] RBP: ffff911e04cd1f82 R08: 0000000000000000 R09:
0000000000000000 
[  984.515588] R10: ffff9c1f007b3bf8 R11: ffffffff8bd711a8 R12:
ffff911e04cd1f7f 
[  984.524474] R13: 0000000000000001 R14: 0000000000000003 R15:
ffff911e009b19c0 
[  984.532878] FS:  00007fbabb42b180(0000) GS:ffff911f37c00000(0000)
knlGS:0000000000000000 
[  984.542084] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[  984.548804] CR2: 00007fbabb7ef000 CR3: 000000014e296001 CR4:

00000000001706f0

aarch64 (arm64):

[root@localhost stress-ng]# ./stress-ng --procfs 0
stress-ng: info: [44802] defaulting to a 86400 second (1 day, 0.00 secs) run per stressor
stress-ng: info: [44802] dispatching hogs: 32 procfs
stress-ng: info: [44802] cache allocate: using defaults, can't determine cache details from sysfs

[ 2934.501319] usercopy: Kernel memory exposure attempt detected from SLUB
object 'kmalloc-128' (offset 82, size 73)! 
[ 2934.516649] ------------[ cut here ]------------ 
[ 2934.524448] kernel BUG at mm/usercopy.c:99! 
[ 2934.532208] Internal error: Oops - BUG: 0 [#1] SMP 
[ 2934.539950] Modules linked in: rfkill sunrpc nicvf cavium_ptp joydev nicpf

cavium_rng_vf thunder_bgx thunder_xcv mdio_thunder cavium_rng mdio_cavium thunderx_edac ipmi_ssif ipmi_devintf ipmi_msghandler vfat fat ip_tables xfs ast i2c_algo_bit drm_vram_helper drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec drm_ttm_helper ttm crct10dif_ce drm ghash_ce gpio_keys i2c_thunderx thunderx_mmc aes_neon_bs

[ 2934.540737] usercopy: Kernel memory exposure attempt detected from SLUB
object 'kmalloc-128' (offset 55, size 108)! 
[ 2934.550255] usercopy: Kernel memory exposure attempt detected from SLUB
object 'kmalloc-128' (offset 86, size 68)! 
[ 2934.550297] ------------[ cut here ]------------ 
[ 2934.550300] kernel BUG at mm/usercopy.c:99! 
[ 2934.589488] CPU: 6 PID: 44874 Comm: stress-ng-procf Not tainted 5.10.0-rc1
#1 
[ 2934.589492] Hardware name: GIGABYTE R120-T34-00/MT30-GS2-00, BIOS F02
08/06/2019 
[ 2934.589497] pstate: 40400005 (nZcv daif +PAN -UAO -TCO BTYPE=--) 
[ 2934.589507] pc : usercopy_abort+0x98/0x9c 
[ 2934.589511] lr : usercopy_abort+0x98/0x9c 
[ 2934.589518] sp : ffff80007e14bc70 
[ 2934.603274] ------------[ cut here ]------------ 
[ 2934.616904] x29: ffff80007e14bc80 x28: ffff00013a6d2a80  
[ 2934.624799] kernel BUG at mm/usercopy.c:99! 
[ 2934.707971]  
[ 2934.707977] x27: 0000000000000000 x26: 0000000000000000  
[ 2934.721063] x25: ffff80007e14bd30 x24: 0000000000000000  
[ 2934.729652] x23: ffff000101e8a540 x22: ffff000149809a9b  
[ 2934.738181] x21: 0000000000000001 x20: 0000000000000049  
[ 2934.746646] x19: ffff000149809a52 x18: 0000000000000000  
[ 2934.755018] x17: 0000000000000000 x16: 0000000000000000  
[ 2934.763363] x15: 0000000000aaaaaa x14: 0000000000000020  
[ 2934.771682] x13: 00000000000117ca x12: ffff8000120bbe00  
[ 2934.780114] x11: 0000000000000003 x10: ffff80001208be18  
[ 2934.788496] x9 : ffff8000102310c0 x8 : ffff80001208bdc0  
[ 2934.796849] x7 : 0000000000000001 x6 : 0000000000000000  
[ 2934.804954] x5 : 0000000000000000 x4 : ffff000ff63af410  
[ 2934.813101] x3 : ffff000ff63be340 x2 : ffff000ff63af410  
[ 2934.821102] x1 : ffff00013a6d2a80 x0 : 0000000000000066  
[ 2934.829282] Call trace: 
[ 2934.834432]  usercopy_abort+0x98/0x9c 
[ 2934.840907]  __check_heap_object+0x124/0x138 
[ 2934.847889]  __check_object_size+0x190/0x210 
[ 2934.854815]  proc_sys_call_handler+0x154/0x220 
[ 2934.861877]  proc_sys_read+0x1c/0x28 
[ 2934.868165]  new_sync_read+0xdc/0x158 
[ 2934.874521]  vfs_read+0x150/0x1e0 
[ 2934.880382]  ksys_read+0x60/0xe8 
[ 2934.886226]  __arm64_sys_read+0x24/0x30 
[ 2934.892799]  el0_svc_common.constprop.0+0xac/0x1e0 
[ 2934.900356]  do_el0_svc+0x2c/0x98 
[ 2934.906348]  el0_sync_handler+0xb0/0xb8 
[ 2934.912797]  el0_sync+0x178/0x180 
[ 2934.918675] Code: aa0003e3 f0002620 911f8000 97fff564 (d4210000)  
[ 2934.927269] ---[ end trace ed6d63c40907130f ]---
Signature · 2 lines
You are receiving this mail because:
You are the assignee for the bug.
#2 lore raw
Quoted 14 lines · Andrew Morton

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 9 lines · On Wed, 28 Oct 2020 15:49:15 +0000 [email protected] wrote:
[  984.279683] Call Trace:
[  984.282581]  __check_heap_object+0xe0/0x110
[  984.287405]  __check_object_size+0x136/0x150
[  984.292347]  proc_sys_call_handler+0x167/0x250
[  984.297565]  new_sync_read+0x108/0x180
[  984.302082]  vfs_read+0x174/0x1d0
[  984.306126]  ksys_read+0x58/0xd0
[  984.310022]  do_syscall_64+0x33/0x40
[  984.314277]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Quoted 1 line

Can we determine which /proc/sys entries these are?

Colin Ian King narrowed it down to
/proc/sys/kernel/sched_domain/cpu0/domain0/flags
and commit 5b9f8ff7b320a34af3dbcf04edb40d9b04f22f4a.

He has a proposed patch, too, that he'll be sending to the list soon. See https://bugzilla.kernel.org/show_bug.cgi?id=209919#c9

Thanks for the quick debugging, Colin!

Signature · 3 lines
Jeff Bastian
Kernel QE - Hardware Enablement
Red Hat