: mm : mmap.c Arithmetic overflow in may_expand_vm()
Subsystems (2)
PATCH 1/1 : mm : mmap.c Arithmetic overflow in may_expand_vm()
The function may_expand_vm() may return a false positive if the proposed increment (npages) is sufficient large to overflow the expression:
cur + npages
Assuming that cur < lim is an invariant, the proposed patch re-arranges the expression to avoid unsigned arithmetic overflow.
More robustly:
if (cur > lim || npages > lim - cur) return 0;
might be preferred if it cannot be guaranteed that cur < lim.
mm/mmap.c renamed from mm/mmap.c.orig +1 -1
@@ -2303,7 +2303,7 @@ int may_expand_vm(struct mm_struct *mm, lim = current->signal->rlim[RLIMIT_AS ].rlim_cur >>PAGE_SHIFT ; -if (cur + npages > lim) +if (npages > lim - cur)return 0 ;return 1 ; }