Skip to contentSkip to conversation

shmdt cannot detach not-alined shm segment cleanly.

PATCH New KAMEZAWA Hiroyuki 1 participant 1 message Feb 9, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 shmdt cannot detach not-alined shm segment cleanly.

This is a patch for a problem of shmdt().
It is found by a test, not a serious problem.
-- Kame

sys_shmdt() can manages shm segments which is covered by multiple vmas. (This can happen when a user uses mprotect() after shmat().)

This works well if shm is aligned to PAGE_SIZE, but if not, the last segment cannot be detached. It is because a comparison in sys_shmdt()

	(vma->vm_end - addr) < size
		addr == return address of shmat()
		size == shmsize, argments to shmget()

size should be aligned to PAGE_SIZE before being compared with vma->vm_end, which is aligned.

ipc/shm.c +1 -0
Index: linux-2.6.16-rc2.org/ipc/shm.c
===================================================================
@@ -870,6 +870,7 @@ asmlinkage long sys_shmdt(char __user *s
 	 * could possibly have landed at. Also cast things to loff_t to
 	 * prevent overflows and make comparisions vs. equal-width types.
 	 */
+	size = PAGE_ALIGN(size);
 	while (vma && (loff_t)(vma->vm_end - addr) <= size) {
 		next = vma->vm_next;