Skip to contentSkip to conversation

mmap for /proc/vmcore broken since 3.12-rc1

DISCUSSION Michael Holzheu 3 participants 7 messages Oct 2, 2013 – Oct 14, 2013

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

Hello Alexey,

Looks like the following commit broke mmap for /proc/vmcore:

commit c4fe24485729fc2cbff324c111e67a1cc2f9adea

Author: Alexey Dobriyan <[email protected]>
Date:   Tue Aug 20 22:17:24 2013 +0300

sparc: fix PCI device proc file mmap(2)

Because /proc/vmcore (fs/proc/vmcore.c) does not implement the get_unmapped_area() fops function mmap now always returns EIO.

Michael

#1 lore raw
Quoted 8 lines · Michael Holzheu

Hello Alexey,

Looks like the following commit broke mmap for /proc/vmcore:

commit c4fe24485729fc2cbff324c111e67a1cc2f9adea

Author: Alexey Dobriyan <[email protected]>
Date:   Tue Aug 20 22:17:24 2013 +0300
Quoted 7 lines

sparc: fix PCI device proc file mmap(2)

Because /proc/vmcore (fs/proc/vmcore.c) does not implement the get_unmapped_area() fops function mmap now always returns EIO.

Michael

I confirmed the bug on v3.12-rc3. According to makedumpfile's log, mmap failed on /proc/vmcore.

mem_map (271)

  mem_map    : ffffea001da40000
  pfn_start  : 878000
  pfn_end    : 880000

Kernel can't mmap vmcore, using reads.
STEP [Excluding unnecessary pages] : 1.268799 seconds STEP [Excluding unnecessary pages] : 1.268756 seconds
STEP [Copying data ] : 44.847924 seconds
Writing erase info...

I'll post a patch later.

Signature · 2 lines
Thanks.
HATAYAMA, Daisuke
#2 lore raw
Quoted 24 lines · HATAYAMA Daisuke
Quoted 8 lines · Michael Holzheu

Hello Alexey,

Looks like the following commit broke mmap for /proc/vmcore:

commit c4fe24485729fc2cbff324c111e67a1cc2f9adea

Author: Alexey Dobriyan <[email protected]>
Date:   Tue Aug 20 22:17:24 2013 +0300
Quoted 7 lines

sparc: fix PCI device proc file mmap(2)

Because /proc/vmcore (fs/proc/vmcore.c) does not implement the get_unmapped_area() fops function mmap now always returns EIO.

Michael

I confirmed the bug on v3.12-rc3. According to makedumpfile's log, mmap failed on /proc/vmcore.

mem_map (271)

   mem_map    : ffffea001da40000
   pfn_start  : 878000
   pfn_end    : 880000
Quoted 8 lines

Kernel can't mmap vmcore, using reads.
STEP [Excluding unnecessary pages] : 1.268799 seconds STEP [Excluding unnecessary pages] : 1.268756 seconds
STEP [Copying data ] : 44.847924 seconds
Writing erase info...

I'll post a patch later.

I've not completed this. I thought it was short task but after I tried to fix, makedumpfile became frequently failing with -ENOMEM and I'm not sure why even now.

Here's current progress.

First, on v3.12-rc3 mmap() on /proc/vmcore fails while returning -EIO. This is due to the commit c4fe24485729fc2cbff324c111e67a1cc2f9adea, just as reported by Holzheu, where proc_reg_get_unmapped_area was newly added to proc_reg_file_ops_no_compat file operations as get_unmapped_area method. Looking at get_unmapped_area function, it calls current->mm->get_unmapped_area at default, but calls f_ops->get_unmapped_area_function if it's assigned.

        get_area = current->mm->get_unmapped_area;
        if (file && file->f_op && file->f_op->get_unmapped_area)
                get_area = file->f_op->get_unmapped_area;
        addr = get_area(file, addr, len, pgoff, flags);
        if (IS_ERR_VALUE(addr))
                return addr;

For regular files in procfs, proc_reg_file_ops_no_compat is used first and then this behaves as wrapper.

static unsigned long proc_reg_get_unmapped_area(struct file *file, unsigned long orig_addr, unsigned long len, unsigned long pgoff, unsigned long flags)

{
        struct proc_dir_entry *pde = PDE(file_inode(file));
        int rv = -EIO;
        unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
        if (use_pde(pde)) {
                get_unmapped_area = pde->proc_fops->get_unmapped_area;
                if (get_unmapped_area)
                        rv = get_unmapped_area(file, orig_addr, len, pgoff, flags);
                unuse_pde(pde);
        }
        return rv;
}

Since this was added in proc_reg_file_ops_no_compat, proc_reg_get_unmapped_area is used in get_unmapped_area now and it always returns -EIO since proc_vmcore_operations has no get_unmapped_area method now.

So, immediate fix idea is to define get_unmapped_area method in proc_vmcore_operations and to design it so that it just calls current->mm->get_unmapped_area.

---
 fs/proc/vmcore.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
fs/proc/vmcore.c +13 -0
index 9100d69..9583419 100644
@@ -412,10 +412,23 @@ static int mmap_vmcore(struct file *file, struct vm_area_struct *vma)
 }
 #endif
 
+static unsigned long
+get_unmapped_area_vmcore(struct file *filp, unsigned long addr,
+                        unsigned long len, unsigned long pgoff,
+                        unsigned long flags)
+{
+#ifdef CONFIG_MMU
+       return current->mm->get_unmapped_area(filp, addr, len, pgoff, flags);
+#else
+       return -EIO;
+#endif
+}
+
 static const struct file_operations proc_vmcore_operations = {
        .read           = read_vmcore,
        .llseek         = default_llseek,
        .mmap           = mmap_vmcore,
+       .get_unmapped_area = get_unmapped_area_vmcore,
 };
 
 static struct vmcore* __init get_new_element(void)

--
1.8.3.1

However, after applying this patch, makedumpfile now somehow fails returning -ENOMEM frequently. It's about 50/128 on my box.

Searching for where to return -ENOMEM in mmap path by printk debug, I found instance of get_unmapped_area returns kernel-space address:

        get_area = current->mm->get_unmapped_area;
        if (file && file->f_op && file->f_op->get_unmapped_area)
                get_area = file->f_op->get_unmapped_area;
        addr = get_area(file, addr, len, pgoff, flags);
        if (IS_ERR_VALUE(addr))
                return addr;

        if (addr > TASK_SIZE - len)   <---- Here
                return -ENOMEM;

The log is:

kdump:/# cd /mnt/
kdump:/mnt# for ((i=0; i<128; ++i)) ; do
Quoted 2 lines

makedumpfile -f -p -d 31 /proc/vmcore vmcore-pd31
done

The kernel version is not supported.
The created dumpfile may be incomplete. cyclic buffer size has been changed: 65535 => 64512

[   49.462536] addr: 0xffffffff8ef28000
[   49.463686] TASK_SIZE: 0x007ffffffff000
[   49.464952] len: 0x00000000400000

Note that makedumpfile tries to mmap some area in 4MiB size here, get_unmapped_area tries to find some area to cover such requested 4MiB size within user-space address space limit but it returns somehow kernel-space address.

The actual instance of current->mm->get_unmapped_area on my environment is arch_get_unmapped_area_topdown.

Finally, note: I tried to run makedumpfile using mmap on /proc/vmcore 128-times. Then,
- v3.12-rc3 returns -EIO in every case (trivial),
- v3.12-rc3 with the above patch applied returns -ENOMEM at 50/128,
- v3.12-rc3 with commit c4fe24485729fc2cbff324c111e67a1cc2f9adea reverted works well in every case, and
- v3.11.1-201.fc19.x86_64 works well in every case.

So, I suspect procfs wrapper affects arch_get_unmapped_region_topdown?

Any comments are helpful.

Signature · 2 lines
Thanks.
HATAYAMA, Daisuke
#3 lore raw
Quoted 8 lines · HATAYAMA Daisuke
fs/proc/vmcore.c +7 -0
+static unsigned long
+get_unmapped_area_vmcore(struct file *filp, unsigned long addr,
+                        unsigned long len, unsigned long pgoff,
+                        unsigned long flags)
+{
+#ifdef CONFIG_MMU
+       return current->mm->get_unmapped_area(filp, addr, len, pgoff,

flags);

fs/proc/vmcore.c static int mmap_vmcore(struct file *file, struct vm_area_struct *vma) · line 431

Quoted 10 lines
fs/proc/vmcore.c +6 -0
+#else
+       return -EIO;
+#endif
+}
+
 static const struct file_operations proc_vmcore_operations = {
        .read           = read_vmcore,
        .llseek         = default_llseek,
        .mmap           = mmap_vmcore,
+       .get_unmapped_area = get_unmapped_area_vmcore,

I think current->mm->get_unmapped_area should be used by core proc code. ENOMEM bug looks unrelated though.

#4 lore raw

Hello,

Quoted 15 lines · Alexey Dobriyan
Quoted 12 lines · HATAYAMA Daisuke
fs/proc/vmcore.c +7 -0
+static unsigned long
+get_unmapped_area_vmcore(struct file *filp, unsigned long addr,
+                        unsigned long len, unsigned long pgoff,
+                        unsigned long flags)
+{
+#ifdef CONFIG_MMU
+       return current->mm->get_unmapped_area(filp, addr, len, pgoff,

flags);

fs/proc/vmcore.c +4 -0
+#else
+       return -EIO;
+#endif
+}
Quoted 8 lines

fs/proc/vmcore.c static int mmap_vmcore(struct file *file, struct vm_area_struct *vma) · line 431

Quoted 6 lines
fs/proc/vmcore.c +2 -0
+
  static const struct file_operations proc_vmcore_operations = {
         .read           = read_vmcore,
         .llseek         = default_llseek,
         .mmap           = mmap_vmcore,
+       .get_unmapped_area = get_unmapped_area_vmcore,

I think current->mm->get_unmapped_area should be used by core proc code.

What do you actually suggest here? You mean moving this code in proc code? I don't think you suggest so.

Quoted 2 lines

ENOMEM bug looks unrelated though.

Next step I'll do is to look into vm_unmapped_region() that looks for for a region fit to a given mmap request and returns its address. In particular, I'll focus on when vm_unmapped_region() could return kernel-space address.

Signature · 2 lines
Thanks.
HATAYAMA, Daisuke
#5 lore raw
Quoted 27 lines · HATAYAMA Daisuke

Hello,

Quoted 15 lines · Alexey Dobriyan
Quoted 12 lines · HATAYAMA Daisuke
(quoted diff) +7 -0
+static unsigned long
+get_unmapped_area_vmcore(struct file *filp, unsigned long addr,
+                        unsigned long len, unsigned long pgoff,
+                        unsigned long flags)
+{
+#ifdef CONFIG_MMU
+       return current->mm->get_unmapped_area(filp, addr, len, pgoff,

flags);

(quoted diff) +4 -0
+#else
+       return -EIO;
+#endif
+}
Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +2 -0
+
  static const struct file_operations proc_vmcore_operations = {
         .read           = read_vmcore,
         .llseek         = default_llseek,
         .mmap           = mmap_vmcore,
+       .get_unmapped_area = get_unmapped_area_vmcore,

I think current->mm->get_unmapped_area should be used by core proc code.

Quoted 2 lines

What do you actually suggest here? You mean moving this code in proc code? I don't think you suggest so.

Please, try this patch, I don't have kexec setup handy.

fs/proc/inode.c +5 -1
@@ -291,7 +291,11 @@ static unsigned long proc_reg_get_unmapped_area(struct file *file, unsigned long
 	int rv = -EIO;
 	unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
 	if (use_pde(pde)) {
-		get_unmapped_area = pde->proc_fops->get_unmapped_area;
+		get_unmapped_area = current->mm->get_unmapped_area;
+#ifdef CONFIG_MMU
+		if (pde->proc_fops->get_unmapped_area)
+			get_unmapped_area = pde->proc_fops->get_unmapped_area;
+#endif
 		if (get_unmapped_area)
 			rv = get_unmapped_area(file, orig_addr, len, pgoff, flags);
 		unuse_pde(pde);
#6 lore raw
Quoted 39 lines · Alexey Dobriyan
Quoted 27 lines · HATAYAMA Daisuke

Hello,

Quoted 15 lines · Alexey Dobriyan
Quoted 12 lines · HATAYAMA Daisuke
(quoted diff) +7 -0
+static unsigned long
+get_unmapped_area_vmcore(struct file *filp, unsigned long addr,
+                        unsigned long len, unsigned long pgoff,
+                        unsigned long flags)
+{
+#ifdef CONFIG_MMU
+       return current->mm->get_unmapped_area(filp, addr, len, pgoff,

flags);

(quoted diff) +4 -0
+#else
+       return -EIO;
+#endif
+}
Quoted 8 lines

Review

Quoted 6 lines
(quoted diff) +2 -0
+
   static const struct file_operations proc_vmcore_operations = {
          .read           = read_vmcore,
          .llseek         = default_llseek,
          .mmap           = mmap_vmcore,
+       .get_unmapped_area = get_unmapped_area_vmcore,

I think current->mm->get_unmapped_area should be used by core proc code.

Quoted 2 lines

What do you actually suggest here? You mean moving this code in proc code? I don't think you suggest so.

Please, try this patch, I don't have kexec setup handy.

fs/proc/inode.c +0 -0
@@ -291,7 +291,11 @@ static unsigned long proc_reg_get_unmapped_area(struct file *file, unsigned long
  	int rv = -EIO;
  	unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);

fs/proc/inode.c static unsigned long proc_reg_get_unmapped_area(struct file *file, unsigned long · line 301

Quoted 10 lines
fs/proc/inode.c +5 -1
  	if (use_pde(pde)) {
-		get_unmapped_area = pde->proc_fops->get_unmapped_area;
+		get_unmapped_area = current->mm->get_unmapped_area;
+#ifdef CONFIG_MMU
+		if (pde->proc_fops->get_unmapped_area)
+			get_unmapped_area = pde->proc_fops->get_unmapped_area;
+#endif
  		if (get_unmapped_area)
  			rv = get_unmapped_area(file, orig_addr, len, pgoff, flags);
  		unuse_pde(pde);

Slight modification to #ifdef ...

get_unmapped_area = NULL;
#ifdef CONFIG_MMU
get_unmapped_area = current->mm->get_unmapped_area #endif
if (pde->proc_fops->get_unmapped_area)
get_unmapped_area = pde->proc_fops->get_unmapped_area;

And, I found the bug. The variable rv should have been defined as unsigned long. sizeof(int) is 4 bytes but sizeof(long) is 8 bytes at least on x86_64.

The reason why returned value looked like kernel virtual address was due to signed extension performed during conversion from negative 32-bit signed integer to 64-bit unsigned long integer.

Hmm, I first checked signature of related functions but overlooked...

Anyway, I'll post fixing patch soon.

Signature · 2 lines
Thanks.
HATAYAMA, Daisuke