I found a synchronization problem in mm/vmalloc.c
A message that is not in the archive (sent off-list or not yet received): 006d01ca8f8b$b62c8ec0$2285ac40$%[email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 002201ca936a$1fc06780$5f413680$@[email protected]
Quoted 57 lines · Yongseok Koh
Sorry, Mr. Morton.
Even though it is somewhat late, I am doing cc the mailing list.
Thanks.
-----Original Message-----
Quoted 4 lines · On Thu, 7 Jan 2010 20:22:30 +0900 "Yongseok Koh" <[email protected]>…
Dear all,
I___mYongseok Koh in Korea.Thanks for the report.
Please do cc a mailing list when reporting bugs so that everyone else knows what's going on.
Quoted 36 lines
I just got a new message in linux-2.6.28.10 (plz refer to the below)
And, one of my colleagues found that there is a synchronization problem in
mm/vmalloc.cIn
free_unmap_area_noflush(),va->flagsis marked asVM_LAZY_FREEfirst, and thenvmap_lazy_nris increased atomically.But, in
__purge_vmap_area_lazy(), while traversing ofvmap_are_list, nr is counted by checkingVM_LAZY_FREEis set tova->flags.After counting the variable nr, kernel reads
vmap_lazy_nratomically and checks aBUG_ONcondition whether nr is greater thanvmap_lazy_nr.The problem is that, if interrupted right after marking
VM_LAZY_FREE, increment ofvmap_lazy_nrcan be delayed.Consequently,
BUG_ONcondition can be met because nr is counted more thanvmap_lazy_nr.What I mentioned is highly probable when vmalloc/vfree are called frequently.
And my colleagues have verified this scenario by adding delay between marking
VM_LAZY_FREEand increasingvmap_lazy_nrinfree_unmap_area_noflush().
Quoted 5 lines
Quoted 2 lines
Am I right ?
Looks plausible to me and as far as I can tell, current code has the same issue.
Yes, I think it's a good catch.
Quoted 1 line
Wakey wakey, Nick! What makes that
BUG_ON()safe? Notpurge_lockafacit?
No I think it is a bug. I would say that we can just get rid of the BUG_ON now. atomic_t is signed, so it should be OK if it momentarily goes negative (and anyway it's only used in a heuristic).
So, thanks for the report. Would you care to send a patch, or propose another way to fix the problem?
Thanks,
Nick
A message that is not in the archive (sent off-list or not yet received): 1263388030.2818.6.camel@barrios-desktop
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 20100114123328.GA7518@laptop
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 20100118073759.GB10052@laptop
A message that is not in the archive (sent off-list or not yet received): 001c01ca98e2$231d8b10$6958a130$@[email protected]
Minchan Kim
Reviewed-byRe: [PATCH] vmalloc: remove BUG_ON due to racy counting of VM_LAZY_FREE
Quoted 1 line · Yongseok Koh
From: Yongseok Koh <[email protected]>
You don't need above line.
We use "From" when we send patch instead of someone.
Quoted 41 lines
In
free_unmap_area_noflush(),va->flagsis marked asVM_LAZY_FREEfirst, and thenvmap_lazy_nris increased atomically.
But, in__purge_vmap_area_lazy(), while traversing ofvmap_are_list, nr is counted by checkingVM_LAZY_FREEis set tova->flags.
After counting the variable nr, kernel readsvmap_lazy_nratomically and checks aBUG_ONcondition whether nr is greater thanvmap_lazy_nrto preventvmap_lazy_nrfrom being negative.The problem is that, if interrupted right after marking
VM_LAZY_FREE, increment ofvmap_lazy_nrcan be delayed.
Consequently,BUG_ONcondition can be met because nr is counted more thanvmap_lazy_nr.It is highly probable when vmalloc/vfree are called frequently. This scenario have been verified by adding delay between marking
VM_LAZY_FREEand increasingvmap_lazy_nrinfree_unmap_area_noflush().Even the
vmap_lazy_nris for checking high watermark, it never be the strict watermark.
Although theBUG_ONcondition is to preventvmap_lazy_nrfrom being negative,vmap_lazy_nris signed variable.
So, it could go down to negative value temporarily.Consequently, removing the
BUG_ONcondition is proper.A possible
BUG_ONmessage is like the below.kernel BUG at
mm/vmalloc.c:517!
invalid opcode: 0000 [#1] SMP
EIP: 0060:[<c04824a4>] EFLAGS: 00010297 CPU: 3 EIP is at__purge_vmap_area_lazy+0x144/0x150
EAX:ee8a8818EBX:c08e77d4ECX:e7c7ae40EDX:c08e77ec
ESI:000081feEDI:e7c7ae60EBP:e7c7ae64ESP:e7c7ae3c
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068Call Trace: [<c0482ad9>] free_unmap_vmap_area_noflush+0x69/0x70 [<c0482b02>] remove_vm_area+0x22/0x70 [<c0482c15>] __vunmap+0x45/0xe0 [<c04831ec>] vmalloc+0x2c/0x30 Code: 8d 59 e0 eb 04 66 90 89 cb 89 d0 e8 87 fe ff ff 8b 43 20 89 da 8d 48
Quoted 6 lines
e0 8d 43 20 3b 04 24 75 e7 fe 05 a8 a5 a3 c0 e9 78 ff ff ff <0f> 0b eb fe 90 8d b4 26 00 00 00 00 56 89 c6 b8 ac a5 a3 c0 31
EIP: [<c04824a4>]__purge_vmap_area_lazy+0x144/0x150SS:ESP 0068:e7c7ae3cSigned-off-by Yongseok Koh <[email protected]>
We discussed about this following as.
http://marc.info/?l=linux-kernel&m=126335856228090&w=2
Thanks for contribution for linux kernel, Yongseok. :)
Signature · 2 lines
Kind regards, Minchan Kim