Skip to contentSkip to conversation

mmap'ed memory in core files ?

PATCH In review Philippe De Muyter 10 participants 25 messages Jul 1, 2008 – Jul 4, 2008

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

		PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Best regards

Philippe

#1 lore raw
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

                PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

Cheers,

Michael

#2 lore raw
Quoted 21 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

                PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

Interesting (and somewhat off topic to your conversation here) - it appears that when dumping mappings, the kernel ignores the maximum core size set with "limit".

This is particularly interesting on a 64 bit kernel where a bug in your code causes you to try to read something about 2Gb into your alleged mmaped file (actual size ~500 bytes) and the segfault causes a coredump.

Bron.

#3 lore raw
Quoted 26 lines · Bron Gondwana
Quoted 21 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

                PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

Quoted 8 lines

Interesting (and somewhat off topic to your conversation here) - it appears that when dumping mappings, the kernel ignores the maximum core size set with "limit".

This is particularly interesting on a 64 bit kernel where a bug in your code causes you to try to read something about 2Gb into your alleged mmaped file (actual size ~500 bytes) and the segfault causes a coredump.

Do you have a ssimple example program for this?

Signature · 4 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
#4 lore raw
Quoted 6 lines
Quoted 4 lines

This is particularly interesting on a 64 bit kernel where a bug in your code causes you to try to read something about 2Gb into your alleged mmaped file (actual size ~500 bytes) and the segfault causes a coredump.

Do you have a ssimple example program for this?

Trying to reproduce the problem, I think it's actually related to sparse files.

$ cat a.c
#include <stdlib.h>
int main() {
 unsigned int i;
 char * a = malloc(3000000000u);
 *(int *)0 = 0;
}
$ gcc a.c
$ ulimit -c 10240 && ./a.out
$ ls -l
-rw------- 1 root root 3000082432 Jul  2 02:23 core.7761
$

It's clearly sparse, but slightly unintuitive that the ulimit doesn't actually limit the filesize, just the size of the data written to the file.

If I change the code to include this line after the malloc():

for (i = 0; i < 3000000000u; i++) a[i] = i % 256;

I get:

-rw------- 1 root root 10485760 Jul 2 02:25 core.8992

More what you'd expect.

One interesting side effect of running a 64-bit kernel + 32-bit userland is that previously bugs that might have previously caused malloc() to fail (eg underflowing integer to a huge value), now succeed and allocate a huge chunk of memory rather than failing and causing the program to bailout/crash on dereference.

Rob

#5 lore raw
Quoted 4 lines · Rob Mueller

It's clearly sparse, but slightly unintuitive that the ulimit doesn't actually limit the filesize, just the size of the data written to the file.

It's the only sane semantic. Imagine ulimit would limit the address range as you seem to be asking for. This means if you set e.g. ulimit -c 1G then the kernel would never dump any address (mmap or not) above 1GB. Never dumping the process stack for example. Clearly doesn't make any sense. And mmap'ed files are not different from any other mappings in this regard.

-Andi

#6 lore raw
Quoted 6 lines · Andi Kleen
Quoted 4 lines · Rob Mueller

It's clearly sparse, but slightly unintuitive that the ulimit doesn't actually limit the filesize, just the size of the data written to the file.

Quoted 6 lines

It's the only sane semantic. Imagine ulimit would limit the address range as you seem to be asking for. This means if you set e.g. ulimit -c 1G then the kernel would never dump any address (mmap or not) above 1GB. Never dumping the process stack for example. Clearly doesn't make any sense. And mmap'ed files are not different from any other mappings in this regard.

Hmm.. the IO hit we got with those Cyrus crashes suggested that it was more than just a small amount of IO being caused. I didn't check if the files were sparse or not.

I guess we can probably create a test case that recreates it, even if it's just running up a dodgy Cyrus instance on the machine that had the crashes before and deliberately recreating a broken cache file to trigger it.

Bron.

#7 lore raw
In reply to Michael Kerrisk

Hi Michael,

Quoted 21 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

                PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Is that a design decision, or a mere finding of the way it is implemented now ?

So, back to my original question :

Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ?

best regards

Philippe

#8 lore raw

[CC+= [email protected]]

Quoted 43 lines · Philippe De Muyter

Hi Michael,

Quoted 21 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

                PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Quoted 8 lines

Is that a design decision, or a mere finding of the way it is implemented now ?

So, back to my original question :

Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ?

Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.

Cheers,

Michael

Signature · 4 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
#9 lore raw

[CC+= [email protected]]

Quoted 66 lines · Michael Kerrisk

[CC+= [email protected]]

Quoted 43 lines · Philippe De Muyter

Hi Michael,

Quoted 21 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

                PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Quoted 8 lines

Is that a design decision, or a mere finding of the way it is implemented now ?

So, back to my original question :

Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ?

Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.

Cheers,

Michael

Signature · 4 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
Signature · 2 lines
Philippe De Muyter  phdm at macqel dot be  Tel +32 27029044
Macq Electronique SA  rue de l'Aeronef 2  B-1140 Bruxelles  Fax +32 27029077
#10 lore raw
Quoted 14 lines
Quoted 14 lines · Philippe De Muyter
Quoted 13 lines

Hi Michael,

Quoted 3 lines · Michael Kerrisk

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

This shouldn't be a problem to you as far as I understand. I suppose "memory mapped I/O pages" means registers of PCI devices, mapped into the memory address space.

The DMA buffer which you would like to get included in the core file is normal RAM (I suppose: allocated by the kernel in the kernel's virtual address space, mapped into the application client's address space by mmap(), and also mapped into the FireWire controller's local bus address space for it to write received data into).

FireWire controllers also have memory-mapped I/O regions (per OHCI: at least one region of at least 2048 bytes size). But your process never accesses them, only the kernel drivers do (to start and stop DMA programs and so on --- while the DMA programs and the DMA data buffers live in normal RAM).

Signature · 3 lines
Stefan Richter
-=====-==--- -=== ---=-
http://arcgraph.de/sr/
#11 lore raw
Quoted 30 lines · Stefan Richter
Quoted 20 lines
Quoted 20 lines
Quoted 18 lines · Philippe De Muyter

Hi Michael,

Quoted 3 lines · Michael Kerrisk

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects
traditional
Linux behavior and means that only anonymous memory segments are
dumped.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

This shouldn't be a problem to you as far as I understand. I suppose "memory mapped I/O pages" means registers of PCI devices, mapped into the memory address space.

Quoted 5 lines

The DMA buffer which you would like to get included in the core file is normal RAM (I suppose: allocated by the kernel in the kernel's virtual address space, mapped into the application client's address space by mmap(), and also mapped into the FireWire controller's local bus address space for it to write received data into).

I agree with your analysis, but the sentence takes as exemple 'frame buffer' which I don't think are registers. I have tested with a video1394 client, and I do not get the mmapped video memory in the core file, even with /proc/PID/coredump_filter set to 0xf. So, while I agree it seems technically feasible, it does not seem to be currently implemented :(

Philippe

#12 lore raw
In reply to Michael Kerrisk

Hi,

Quoted 51 lines · Michael Kerrisk

[CC+= [email protected]]

Quoted 47 lines · Philippe De Muyter

Hi Michael,

Quoted 22 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

               PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Is that a design decision, or a mere finding of the way it is implemented now ?

MMIO pages have been not dumped since a long time ago, and I didn't target them for the coredump_filter feature because I thought it was generally danger to read MMIO pages. As for frame buffer we would be able to safely access to it, but there is no way to tell it from other MMIO mappings, AFAIK.

Quoted 8 lines
Quoted 5 lines

So, back to my original question :

Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ?

Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.

The following patch may help. To dump MMIO pages, set bit 5 of coredump_filter.

$ echo 0x23 > /proc/<PID>/coredump_filter

---
This patch is not intended to be merged to the upstream kernel because the safeness of reading VM_IO mappings has not been proved.

fs/binfmt_elf.c +10 -3
Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
===================================================================
@@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
 	if (vma->vm_flags & VM_ALWAYSDUMP)
 		goto whole;
 
-	/* Do not dump I/O mapped devices or special mappings */
-	if (vma->vm_flags & (VM_IO | VM_RESERVED))
+#define FILTER(type)	(mm_flags & (1UL << MMF_DUMP_##type))
+
+	/* By default, do not dump memory mapped I/O mappings */
+	if (vma->vm_flags & VM_IO) {
+		if (FILTER(MMIO))
+			goto whole;
 		return 0;
+	}
 
-#define FILTER(type)	(mm_flags & (1UL << MMF_DUMP_##type))
+	/* Do not dump special mappings */
+	if (vma->vm_flags & VM_RESERVED)
+		return 0;
 
 	/* By default, dump shared memory if mapped from an anonymous file. */
 	if (vma->vm_flags & VM_SHARED) {
include/linux/sched.h +2 -1
Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
===================================================================
@@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
 #define MMF_DUMP_MAPPED_PRIVATE	4
 #define MMF_DUMP_MAPPED_SHARED	5
 #define MMF_DUMP_ELF_HEADERS	6
+#define MMF_DUMP_MMIO		7
 #define MMF_DUMP_FILTER_SHIFT	MMF_DUMPABLE_BITS
-#define MMF_DUMP_FILTER_BITS	5
+#define MMF_DUMP_FILTER_BITS	6
 #define MMF_DUMP_FILTER_MASK \
 	(((1 << MMF_DUMP_FILTER_BITS) - 1) << MMF_DUMP_FILTER_SHIFT)
 #define MMF_DUMP_FILTER_DEFAULT \

Regards,

Signature · 3 lines
Hidehiro Kawai
Hitachi, Systems Development Laboratory
Linux Technology Center
#13 lore raw

Hello Hidehiro,

Quoted 127 lines · Hidehiro Kawai

Hi,

Quoted 51 lines · Michael Kerrisk

[CC+= [email protected]]

Quoted 47 lines · Philippe De Muyter

Hi Michael,

Quoted 22 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

               PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Is that a design decision, or a mere finding of the way it is implemented now ?

MMIO pages have been not dumped since a long time ago, and I didn't target them for the coredump_filter feature because I thought it was generally danger to read MMIO pages. As for frame buffer we would be able to safely access to it, but there is no way to tell it from other MMIO mappings, AFAIK.

Quoted 8 lines
Quoted 5 lines

So, back to my original question :

Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ?

Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.

The following patch may help. To dump MMIO pages, set bit 5 of coredump_filter.

$ echo 0x23 > /proc/<PID>/coredump_filter

---
This patch is not intended to be merged to the upstream kernel because the safeness of reading VM_IO mappings has not been proved.

Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
===================================================================

fs/binfmt_elf.c +10 -3
@@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
 	if (vma->vm_flags & VM_ALWAYSDUMP)
 		goto whole;
 
-	/* Do not dump I/O mapped devices or special mappings */
-	if (vma->vm_flags & (VM_IO | VM_RESERVED))
+#define FILTER(type)	(mm_flags & (1UL << MMF_DUMP_##type))
+
+	/* By default, do not dump memory mapped I/O mappings */
+	if (vma->vm_flags & VM_IO) {
+		if (FILTER(MMIO))
+			goto whole;
 		return 0;
+	}
 
-#define FILTER(type)	(mm_flags & (1UL << MMF_DUMP_##type))
+	/* Do not dump special mappings */
+	if (vma->vm_flags & VM_RESERVED)
+		return 0;
 
 	/* By default, dump shared memory if mapped from an anonymous file. */
 	if (vma->vm_flags & VM_SHARED) {

Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
===================================================================

include/linux/sched.h +2 -1
@@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
 #define MMF_DUMP_MAPPED_PRIVATE	4
 #define MMF_DUMP_MAPPED_SHARED	5
 #define MMF_DUMP_ELF_HEADERS	6
+#define MMF_DUMP_MMIO		7
 #define MMF_DUMP_FILTER_SHIFT	MMF_DUMPABLE_BITS
-#define MMF_DUMP_FILTER_BITS	5
+#define MMF_DUMP_FILTER_BITS	6
 #define MMF_DUMP_FILTER_MASK \
 	(((1 << MMF_DUMP_FILTER_BITS) - 1) << MMF_DUMP_FILTER_SHIFT)
 #define MMF_DUMP_FILTER_DEFAULT \
Quoted 1 line

Regards,

Thanks for your patch, but it will not help here. Before applying it blindly I asked myself if the mmapped region was tagged VM_IO, because it is actually a simple ram zone, not an I/O zone, and the answer is it is not a VM_IO zone. Details :

drivers/ieee1394/video1394.c:

    static int video1394_mmap(struct file *file, struct vm_area_struct *vma)
    {
    [...]
    return dma_region_mmap(&ctx->current_ctx->dma, file, vma);
    }

drivers/ieee1394/dma.c:

    int dma_region_mmap(struct dma_region *dma, struct file *file,
			struct vm_area_struct *vma)
    {
    [...]
    vma->vm_ops = &dma_region_vm_ops;
    vma->vm_private_data = dma;
    vma->vm_file = file;
    vma->vm_flags |= VM_RESERVED;

    return 0;
    }

So, actually the zone I would like to get dumped in the core file is tagged VM_RESERVED.

I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.

As I don't know the real meaning of VM_RESERVED, I do not know which choice is the best one for the official kernel tree, but locally I'll go for adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.

Philippe

#14 lore raw

Hello,

Quoted 167 lines · Philippe De Muyter

Hello Hidehiro,

Quoted 132 lines · Hidehiro Kawai

Hi,

Quoted 54 lines · Michael Kerrisk

[CC+= [email protected]]

Quoted 49 lines · Philippe De Muyter

Hi Michael,

Quoted 23 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

              PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Is that a design decision, or a mere finding of the way it is implemented now ?

MMIO pages have been not dumped since a long time ago, and I didn't target them for the coredump_filter feature because I thought it was generally danger to read MMIO pages. As for frame buffer we would be able to safely access to it, but there is no way to tell it from other MMIO mappings, AFAIK.

Quoted 8 lines
Quoted 5 lines

So, back to my original question :

Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ?

Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.

The following patch may help. To dump MMIO pages, set bit 5 of coredump_filter.

$ echo 0x23 > /proc/<PID>/coredump_filter

---
This patch is not intended to be merged to the upstream kernel because the safeness of reading VM_IO mappings has not been proved.

Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
===================================================================

fs/binfmt_elf.c +10 -3
@@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
 	if (vma->vm_flags & VM_ALWAYSDUMP)
 		goto whole;
 
-	/* Do not dump I/O mapped devices or special mappings */
-	if (vma->vm_flags & (VM_IO | VM_RESERVED))
+#define FILTER(type)	(mm_flags & (1UL << MMF_DUMP_##type))
+
+	/* By default, do not dump memory mapped I/O mappings */
+	if (vma->vm_flags & VM_IO) {
+		if (FILTER(MMIO))
+			goto whole;
 		return 0;
+	}
 
-#define FILTER(type)	(mm_flags & (1UL << MMF_DUMP_##type))
+	/* Do not dump special mappings */
+	if (vma->vm_flags & VM_RESERVED)
+		return 0;
 
 	/* By default, dump shared memory if mapped from an anonymous file. */
 	if (vma->vm_flags & VM_SHARED) {

Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
===================================================================

include/linux/sched.h +2 -1
@@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
 #define MMF_DUMP_MAPPED_PRIVATE	4
 #define MMF_DUMP_MAPPED_SHARED	5
 #define MMF_DUMP_ELF_HEADERS	6
+#define MMF_DUMP_MMIO		7
 #define MMF_DUMP_FILTER_SHIFT	MMF_DUMPABLE_BITS
-#define MMF_DUMP_FILTER_BITS	5
+#define MMF_DUMP_FILTER_BITS	6
 #define MMF_DUMP_FILTER_MASK \
 	(((1 << MMF_DUMP_FILTER_BITS) - 1) << MMF_DUMP_FILTER_SHIFT)
 #define MMF_DUMP_FILTER_DEFAULT \
Quoted 1 line

Regards,

Thanks for your patch, but it will not help here. Before applying it blindly I asked myself if the mmapped region was tagged VM_IO, because it is actually a simple ram zone, not an I/O zone, and the answer is it is not a VM_IO zone. Details :

drivers/ieee1394/video1394.c:

    static int video1394_mmap(struct file *file, struct vm_area_struct *vma)
    {
    [...]
    return dma_region_mmap(&ctx->current_ctx->dma, file, vma);
    }

drivers/ieee1394/dma.c:

    int dma_region_mmap(struct dma_region *dma, struct file *file,
			struct vm_area_struct *vma)
    {
    [...]
    vma->vm_ops = &dma_region_vm_ops;
    vma->vm_private_data = dma;
    vma->vm_file = file;
    vma->vm_flags |= VM_RESERVED;

    return 0;
    }

So, actually the zone I would like to get dumped in the core file is tagged VM_RESERVED.

Quoted 8 lines

I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.

As I don't know the real meaning of VM_RESERVED, I do not know which choice is the best one for the official kernel tree, but locally I'll go for adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.

I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either. Allowing everyone to choose whether dump the dma region or not, perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and a coredump_filter bit which controls (VM_IO | VM_RESERVED) && VM_DUMPABLE area, for example.

I think it is also OK to just add VM_ALWAYSDUMP flag to the dma region if the device driver knows the region is safely readable and small enough.

Regards,

Signature · 3 lines
Hidehiro Kawai
Hitachi, Systems Development Laboratory
Linux Technology Center
#15 lore raw
Quoted 17 lines · Hidehiro Kawai
Quoted 16 lines · Philippe De Muyter

drivers/ieee1394/dma.c:

    int dma_region_mmap(struct dma_region *dma, struct file *file,
			struct vm_area_struct *vma)
    {
    [...]
    vma->vm_ops = &dma_region_vm_ops;
    vma->vm_private_data = dma;
    vma->vm_file = file;
    vma->vm_flags |= VM_RESERVED;

    return 0;
    }

So, actually the zone I would like to get dumped in the core file is tagged VM_RESERVED.

Quoted 7 lines
Quoted 7 lines

I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.

As I don't know the real meaning of VM_RESERVED, I do not know which choice is the best one for the official kernel tree,

I don't know these things either. But among else, VM_RESERVED prevents a vma from being swapped out. Makes kind of sense, given that besides the CPUs also the FireWire controller writes to this memory.

Quoted 9 lines
Quoted 2 lines

but locally I'll go for
adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.

I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either. Allowing everyone to choose whether dump the dma region or not, perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and a coredump_filter bit which controls (VM_IO | VM_RESERVED) && VM_DUMPABLE area, for example.

Quoted 3 lines

I think it is also OK to just add VM_ALWAYSDUMP flag to the dma region if the device driver knows the region is safely readable and small enough.

It is safely readable. I don't know if it is small enough. The size of the DMA buffer is AFAIK chosen by userspace (by the application program or maybe a library) which uses the character device file ABIs for isochronous FireWire IO of raw1394, video1394, or dv1394.

Signature · 3 lines
Stefan Richter
-=====-==--- -=== --=--
http://arcgraph.de/sr/
#16 lore raw
Quoted 43 lines · Stefan Richter
Quoted 18 lines · Hidehiro Kawai
Quoted 17 lines · Philippe De Muyter

drivers/ieee1394/dma.c:

    int dma_region_mmap(struct dma_region *dma, struct file *file,
			struct vm_area_struct *vma)
    {
    [...]
    vma->vm_ops = &dma_region_vm_ops;
    vma->vm_private_data = dma;
    vma->vm_file = file;
    vma->vm_flags |= VM_RESERVED;

    return 0;
    }

So, actually the zone I would like to get dumped in the core file is tagged
VM_RESERVED.

Quoted 8 lines
Quoted 8 lines

I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.

As I don't know the real meaning of VM_RESERVED, I do not know which choice
is the best one for the official kernel tree,

I don't know these things either. But among else, VM_RESERVED prevents a vma from being swapped out. Makes kind of sense, given that besides the CPUs also the FireWire controller writes to this memory.

Quoted 10 lines
Quoted 2 lines

but locally I'll go for
adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.

I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either. Allowing everyone to choose whether dump the dma region or not, perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and a coredump_filter bit which controls (VM_IO | VM_RESERVED) && VM_DUMPABLE area, for example.
I think it is also OK to just add VM_ALWAYSDUMP flag to the dma region if the device driver knows the region is safely readable and small enough.

Quoted 4 lines

It is safely readable. I don't know if it is small enough. The size of the DMA buffer is AFAIK chosen by userspace (by the application program or maybe a library) which uses the character device file ABIs for isochronous FireWire IO of raw1394, video1394, or dv1394.

For video1394, the size of the buffer is N times the size of a frame, with
N choosen by the application but small (let say < 32). The size of the
frame depends on the camera. I have already worked with 8bpp 1600x1200 cameras,but with a smaller N in that case.

1600x1200x32 gives 60M. If they were allocated by malloc instead of by mmap, they would be dumped in the core files without any question about the size.

So I think we may consider the 'small enough' criteria is met.

Best regards

Philippe

#17 lore raw
In reply to Stefan Richter
Quoted 15 lines · Stefan Richter
Quoted 5 lines · Hidehiro Kawai
Quoted 4 lines · Philippe De Muyter

As I don't know the real meaning of VM_RESERVED, I do not know which choice
is the best one for the official kernel tree,

I don't know these things either. But among else, VM_RESERVED prevents a vma from being swapped out. Makes kind of sense, given that besides the CPUs also the FireWire controller writes to this memory.

Quoted 4 lines
Quoted 2 lines

but locally I'll go for
adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.

I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either.

They're rather an embarrassment, nobody can say quite what VM_RESERVED and VM_IO really mean, beyond not-your-ordinary-vma: VM_VOODOO, beware. They both say "count me as reserved_vm", which means /proc/<pid>/status leaves them out of VmSize.

Certainly VM_IO hints at special I/O memory, but in most cases there's nothing very special about either: just memory preallocated by a driver.

In 2.4 VM_RESERVED indeed prevented swapout from looking at the pages of that vma. We probably ought to have killed it as part of the 2.5 switch to rmap. In 2.6 the pages of a VM_RESERVED vma wouldn't be considered for swapout, because they're not put on any LRU for that. I think I promised (ah, that's a shameful phrase!) to get rid of it a few years ago, but never quite got around to it.

VM_RESERVED and VM_IO are treated as equivalent in most tests, but there's just a few places, e.g. get_user_pages, where we're scared off by VM_IO but don't mind VM_RESERVED.

VM_ALWAYSDUMP: marvellous, a flag with an understandable name and a clear purpose.

Hugh

#18 lore raw
In reply to Hidehiro Kawai

Hi Hidehiro,
On Fri, Jul 04, 2008 at 02:50:25PM +0900, Hidehiro Kawai wrote:
[...]

Quoted 15 lines
Quoted 8 lines

I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.

As I don't know the real meaning of VM_RESERVED, I do not know which choice is the best one for the official kernel tree, but locally I'll go for adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.

I'm afraid I don't know real usages of VM_RESERVED and VM_IO, either. Allowing everyone to choose whether dump the dma region or not, perhaps we need to introduce a new VM flag (e.g. VM_DUMPABLE) and a coredump_filter bit which controls (VM_IO | VM_RESERVED) && VM_DUMPABLE area, for example.

Quoted 3 lines

I think it is also OK to just add VM_ALWAYSDUMP flag to the dma region if the device driver knows the region is safely readable and small enough.

I have just submitted a patch doing that for ieee1394 dma regions:

http://marc.info/?l=linux-kernel&m=121510404729225&w=2

Thanks for your help

Philippe

#19 lore raw
In reply to Hidehiro Kawai

[Sorry, resent with CC += [email protected]] Hello Hidehiro,

Quoted 127 lines · Hidehiro Kawai

Hi,

Quoted 51 lines · Michael Kerrisk

[CC+= [email protected]]

Quoted 47 lines · Philippe De Muyter

Hi Michael,

Quoted 22 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

               PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Is that a design decision, or a mere finding of the way it is implemented now ?

MMIO pages have been not dumped since a long time ago, and I didn't target them for the coredump_filter feature because I thought it was generally danger to read MMIO pages. As for frame buffer we would be able to safely access to it, but there is no way to tell it from other MMIO mappings, AFAIK.

Quoted 8 lines
Quoted 5 lines

So, back to my original question :

Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ?

Perhaps Hidehiro, who wrote the coredump_filter feature, can provide insight.

The following patch may help. To dump MMIO pages, set bit 5 of coredump_filter.

$ echo 0x23 > /proc/<PID>/coredump_filter

---
This patch is not intended to be merged to the upstream kernel because the safeness of reading VM_IO mappings has not been proved.

Index: linux-2.6.26-rc5-mm3/fs/binfmt_elf.c
===================================================================

fs/binfmt_elf.c +10 -3
@@ -1141,11 +1141,18 @@ static unsigned long vma_dump_size(struc
 	if (vma->vm_flags & VM_ALWAYSDUMP)
 		goto whole;
 
-	/* Do not dump I/O mapped devices or special mappings */
-	if (vma->vm_flags & (VM_IO | VM_RESERVED))
+#define FILTER(type)	(mm_flags & (1UL << MMF_DUMP_##type))
+
+	/* By default, do not dump memory mapped I/O mappings */
+	if (vma->vm_flags & VM_IO) {
+		if (FILTER(MMIO))
+			goto whole;
 		return 0;
+	}
 
-#define FILTER(type)	(mm_flags & (1UL << MMF_DUMP_##type))
+	/* Do not dump special mappings */
+	if (vma->vm_flags & VM_RESERVED)
+		return 0;
 
 	/* By default, dump shared memory if mapped from an anonymous file. */
 	if (vma->vm_flags & VM_SHARED) {

Index: linux-2.6.26-rc5-mm3/include/linux/sched.h
===================================================================

include/linux/sched.h +2 -1
@@ -403,8 +403,9 @@ extern int get_dumpable(struct mm_struct
 #define MMF_DUMP_MAPPED_PRIVATE	4
 #define MMF_DUMP_MAPPED_SHARED	5
 #define MMF_DUMP_ELF_HEADERS	6
+#define MMF_DUMP_MMIO		7
 #define MMF_DUMP_FILTER_SHIFT	MMF_DUMPABLE_BITS
-#define MMF_DUMP_FILTER_BITS	5
+#define MMF_DUMP_FILTER_BITS	6
 #define MMF_DUMP_FILTER_MASK \
 	(((1 << MMF_DUMP_FILTER_BITS) - 1) << MMF_DUMP_FILTER_SHIFT)
 #define MMF_DUMP_FILTER_DEFAULT \
Quoted 1 line

Regards,

Thanks for your patch, but it will not help here. Before applying it blindly I asked myself if the mmapped region was tagged VM_IO, because it is actually a simple ram zone, not an I/O zone, and the answer is it is not a VM_IO zone. Details :

drivers/ieee1394/video1394.c:

    static int video1394_mmap(struct file *file, struct vm_area_struct *vma)
    {
    [...]
    return dma_region_mmap(&ctx->current_ctx->dma, file, vma);
    }

drivers/ieee1394/dma.c:

    int dma_region_mmap(struct dma_region *dma, struct file *file,
			struct vm_area_struct *vma)
    {
    [...]
    vma->vm_ops = &dma_region_vm_ops;
    vma->vm_private_data = dma;
    vma->vm_file = file;
    vma->vm_flags |= VM_RESERVED;

    return 0;
    }

So, actually the zone I would like to get dumped in the core file is tagged VM_RESERVED.

I see the following ways to solve my problem :
- do not tag the zone as VM_RESERVED in ieee1394::dma_region_mmap
- tag the zone as VM_ALWAYSDUMP in ieee1394::dma_region_mmap
- add a bit in coredump_filter to dump the VM_RESERVED zones.

As I don't know the real meaning of VM_RESERVED, I do not know which choice is the best one for the official kernel tree, but locally I'll go for adding VM_ALWAYSDUMP in ieee1394::dma_region_mmap.

Philippe

#20 lore raw

PATCH 1/1 ieee1394 : dump mmapped video1394 buffers in core files

Currently, core files do not contain the mmapped memory of the video1394 or dv1394 devices, which contain the actual video input, making it impossible to analyse the cause of abnormal program termination for image analysis or (de)compression software. Fix that.

---

drivers/ieee1394/dma.c +1 -1
diff -r ced66ca0044f drivers/ieee1394/dma.c
@@ -274,7 +274,7 @@ int dma_region_mmap(struct dma_region *d
 	vma->vm_ops = &dma_region_vm_ops;
 	vma->vm_private_data = dma;
 	vma->vm_file = file;
-	vma->vm_flags |= VM_RESERVED;
+	vma->vm_flags |= VM_RESERVED | VM_ALWAYSDUMP;
 
 	return 0;
 }
#21 lore raw

Re: [PATCH] ieee1394 : dump mmapped video1394 buffers in core files

Quoted 2 lines · Philippe De Muyter

Currently, core files do not contain the mmapped memory of the video1394 or dv1394 devices,

Ditto with /dev/raw1394 when the rawiso ioctl API is used.

Quoted 10 lines

which contain the actual video input, making it impossible to analyse the cause of abnormal program termination for image analysis or (de)compression software. Fix that.

---
diff -r ced66ca0044f drivers/ieee1394/dma.c

drivers/ieee1394/dma.c int dma_region_mmap(struct dma_region *d · line 280

Quoted 8 lines
drivers/ieee1394/dma.c +1 -1
@@ -274,7 +274,7 @@ int dma_region_mmap(struct dma_region *d
 	vma->vm_ops = &dma_region_vm_ops;
 	vma->vm_private_data = dma;
 	vma->vm_file = file;
-	vma->vm_flags |= VM_RESERVED;
+	vma->vm_flags |= VM_RESERVED | VM_ALWAYSDUMP;
 
 	return 0;
 }

I'll commit it with the title changed to "dump mmapped iso buffers in core files" if that's OK.

Signature · 3 lines
Stefan Richter
-=====-==--- -=== --=--
http://arcgraph.de/sr/
#22 lore raw

Re: [PATCH] ieee1394 : dump mmapped video1394 buffers in core files

Quoted 17 lines · Stefan Richter
Quoted 2 lines · Philippe De Muyter

Currently, core files do not contain the mmapped memory of the video1394 or dv1394 devices,

Ditto with /dev/raw1394 when the rawiso ioctl API is used.

Quoted 11 lines

which contain the actual video input, making it impossible to analyse the cause of abnormal program termination for image analysis or (de)compression software. Fix that.

---
diff -r ced66ca0044f drivers/ieee1394/dma.c

drivers/ieee1394/dma.c +0 -0
@@ -274,7 +274,7 @@ int dma_region_mmap(struct dma_region *d
 	vma->vm_ops = &dma_region_vm_ops;
 	vma->vm_private_data = dma;
Quoted 8 lines

drivers/ieee1394/dma.c int dma_region_mmap(struct dma_region *d · line 280

Quoted 5 lines
drivers/ieee1394/dma.c +1 -1
 	vma->vm_file = file;
-	vma->vm_flags |= VM_RESERVED;
+	vma->vm_flags |= VM_RESERVED | VM_ALWAYSDUMP;
  	return 0;
 }

I'll commit it with the title changed to "dump mmapped iso buffers in core files" if that's OK.

OK for me

Philippe

#23 lore raw
In reply to Philippe De Muyter
Quoted 7 lines · Philippe De Muyter

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Is that a design decision, or a mere finding of the way it is implemented now ?

It's an implementation issue, and there are patches queue for 2.6.27 that allow gdb access to these regions, which should be easily extendable to cover core dumps, too.

#24 lore raw
In reply to Michael Kerrisk

[ sorry, reposted to add cc [email protected] ]

Hi Michael,

Quoted 21 lines · Michael Kerrisk
Quoted 10 lines · Philippe De Muyter

Hello everybody,

I develop video acquisition software using the video1394 interface. The images grabbed by the camera and iee1394 bus are kept in kernel memory and made available to the user program through a mmap call done in the libdc1394 library :

dma_ring_buffer= mmap(0, vmmap.nb_buffers * vmmap.buf_size,

                PROT_READ|PROT_WRITE,MAP_SHARED, craw->capture.dma_fd, 0);
Quoted 8 lines

Sometimes, my program crashes and produces a core file :) It seems to me that the core file does not contain the mmap'ed memory and hence I cannot replay my program with the same image for debugging purpose.

Is it possible to configure the kernel through /proc, or through the mmap system call to have that mmapped segment in the core file, or do I need to modify the kernel itself to obtain the behaviour I want ? If I need to modify the kernel, can some kind soul provide me some pointers ?

Quoted 3 lines

Have a look at the section "Controlling which mappings are written to the core dump" in a recent core.5 man page:
http://www.kernel.org/doc/man-pages/online/pages/man5/core.5.html

thanks for the info. I didn't know about /proc/PID/coredump_filter.

that part was promising :

bit 2 Dump file-backed private mappings.
bit 3 Dump file-backed shared mappings.

The default value of coredump_filter is 0x3; this reflects traditional Linux behavior and means that only anonymous memory segments are dumped.

Unfortunately, the part that applies to me (I have tested it) is the next one :

Memory-mapped I/O pages such as frame buffer are never dumped, [...], regardless of the coredump_filter value.

Is that a design decision, or a mere finding of the way it is implemented now ?

So, back to my original question :

Can some kind soul provide me some pointers to the way I should modify the kernel to make the inclusion of the video1394 mmapped segment in core files possible ?

best regards

Philippe