Skip to contentSkip to conversation

Non disruptive application core dump infrastructure

DISCUSSION Janani Venkataraman 3 participants 10 messages May 29, 2014 – Jul 30, 2014 Util-Linux package

lore
Share & tools
lore
#0 lore raw

Hi,

We have developed a tool called "gencore" which captures the core of an application without disrupting its process. The dump is collected non-disruptively and this tool currently supports s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore" which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This is implemented through a daemon which listens on a UNIX Filesocket for such requests. The daemon is started immediately post installation. The program which requires the dump makes use of the gencore() API and provides the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */ lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel community. The patches have been posted
at: https://lkml.org/lkml/2014/3/20/138

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

#1 lore raw
Quoted 44 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of an application without disrupting its process. The dump is collected non-disruptively and this tool currently supports s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore" which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This is implemented through a daemon which listens on a UNIX Filesocket for such requests. The daemon is started immediately post installation. The program which requires the dump makes use of the gencore() API and provides the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */ lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel community. The patches have been posted
at: https://lkml.org/lkml/2014/3/20/138

Quoted 11 lines

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe util-linux" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Interesting,
but how is this different from attaching to a process with GDB and using the gcore command? Or to automate it more, using the gcore script that comes with GDB? Cheers,
Ondrej

#2 lore raw
Quoted 67 lines · Ondrej Oprala
Quoted 54 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of an application without
disrupting its process. The dump is collected non-disruptively and this tool currently supports
s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore"
which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to
be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This
is implemented through a daemon which listens on a UNIX Filesocket for such requests. The daemon is started
immediately post installation. The program which requires the dump makes use of the gencore() API and provides
the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */
lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the
ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel community. The patches have been posted
at: https://lkml.org/lkml/2014/3/20/138

Quoted 12 lines

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe util-linux" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Quoted 7 lines

Interesting,
but how is this different from attaching to a process with GDB and using the gcore command? Or to automate it more, using the gcore script that comes with GDB?
Cheers,
Ondrej

There are two major issues with that.

1) GDB uses PTRACE_ATTACH and hence the process gets a SIGSTOP.

2) A process cannot initiate the request to dump itself, say from a signal handler. (since fork() is not signal safe)

Thanks
Suzuki

#3 lore raw
Quoted 75 lines · Suzuki K. Poulose
Quoted 67 lines · Ondrej Oprala
Quoted 54 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of an application without
disrupting its process. The dump is collected non-disruptively and this tool currently supports
s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore"
which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to
be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This
is implemented through a daemon which listens on a UNIX Filesocket for such requests. The daemon is started
immediately post installation. The program which requires the dump makes use of the gencore() API and provides
the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */
lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the
ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel community. The patches have been posted
at: https://lkml.org/lkml/2014/3/20/138

Quoted 12 lines

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe util-linux" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Quoted 7 lines

Interesting,
but how is this different from attaching to a process with GDB and using the gcore command? Or to automate it more, using the gcore script that comes with GDB?
Cheers,
Ondrej

Quoted 3 lines

There are two major issues with that.

1) GDB uses PTRACE_ATTACH and hence the process gets a SIGSTOP.

I fail to see the downside to that.

Quoted 2 lines

2) A process cannot initiate the request to dump itself, say from a signal handler. (since fork() is not signal safe)

This should be possible using libgdb. Let's say forking while in a SIGSEGV handler and using the libgdb API to do the dump.

Quoted 7 lines

Thanks
Suzuki

--
To unsubscribe from this list: send the line "unsubscribe util-linux" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Thanks,
Ondrej

#4 lore raw
Quoted 77 lines · Ondrej Oprala
Quoted 76 lines · Suzuki K. Poulose
Quoted 75 lines · Ondrej Oprala
Quoted 54 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of an application without
disrupting its process. The dump is collected non-disruptively and this tool currently supports
s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore"
which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to
be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This
is implemented through a daemon which listens on a UNIX Filesocket for such requests. The daemon is started
immediately post installation. The program which requires the dump makes use of the gencore() API and provides
the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */
lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the
ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel community. The patches have been posted
at: https://lkml.org/lkml/2014/3/20/138

Quoted 13 lines

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe
util-linux" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Interesting,
but how is this different from attaching to a process with GDB and using the gcore command? Or to automate it more, using the gcore script that comes with GDB?
Cheers,
Ondrej

Quoted 8 lines
Quoted 3 lines

There are two major issues with that.

1) GDB uses PTRACE_ATTACH and hence the process gets a SIGSTOP.

I fail to see the downside to that.

Quoted 2 lines

2) A process cannot initiate the request to dump itself, say from a signal handler. (since fork() is not signal safe)

This should be possible using libgdb. Let's say forking while in a SIGSEGV handler and using the libgdb API to do the dump.

Thats exactly the problem. forking within a sighandler is not safe. You could possibly deadlock with glibc locks.

Thanks
Suzuki

Quoted 10 lines
Quoted 7 lines

Thanks
Suzuki

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe util-linux" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thanks,
Ondrej

#5 lore raw
Quoted 80 lines · Suzuki K. Poulose
Quoted 79 lines · Ondrej Oprala
Quoted 78 lines · Suzuki K. Poulose
Quoted 68 lines · Ondrej Oprala
Quoted 54 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of an application without
disrupting its process. The dump is collected non-disruptively and this tool currently supports
s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore"
which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to
be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This
is implemented through a daemon which listens on a UNIX Filesocket for such requests. The daemon is started
immediately post installation. The program which requires the dump makes use of the gencore() API and provides
the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */
lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the
ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel community. The patches have been posted
at: https://lkml.org/lkml/2014/3/20/138

Quoted 13 lines

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe
util-linux" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Quoted 7 lines

Interesting,
but how is this different from attaching to a process with GDB and using the gcore command? Or to automate it more, using the gcore script that comes with GDB?
Cheers,
Ondrej

There are two major issues with that.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

1) GDB uses PTRACE_ATTACH and hence the process gets a SIGSTOP.

I fail to see the downside to that.

Quoted 2 lines

2) A process cannot initiate the request to dump itself, say from a signal handler. (since fork() is not signal safe)

This should be possible using libgdb. Let's say forking while in a SIGSEGV handler and using the libgdb API to do the dump.

Thats exactly the problem. forking within a sighandler is not safe. You could possibly deadlock with glibc locks.

Ondrej,

What are your thoughts about this ?

Thanks
Suzuki

#6 lore raw
Quoted 87 lines · Suzuki K. Poulose
Quoted 86 lines · Suzuki K. Poulose
Quoted 85 lines · Ondrej Oprala
Quoted 76 lines · Suzuki K. Poulose
Quoted 68 lines · Ondrej Oprala
Quoted 54 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of an application without
disrupting its process. The dump is collected non-disruptively and this tool currently supports
s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore"
which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to
be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This
is implemented through a daemon which listens on a UNIX Filesocket for such requests. The daemon is started
immediately post installation. The program which requires the dump makes use of the gencore() API and provides
the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */
lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the
ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel community. The patches have been posted at:https://lkml.org/lkml/2014/3/20/138

Quoted 13 lines

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe
util-linux" in
the body of a message [email protected]
More majordomo info athttp://vger.kernel.org/majordomo-info.html
Quoted 7 lines

Interesting,
but how is this different from attaching to a process with GDB and using the gcore command? Or to automate it more, using the gcore script that comes with GDB?
Cheers,
Ondrej

Quoted 3 lines

There are two major issues with that.

1) GDB uses PTRACE_ATTACH and hence the process gets a SIGSTOP.

I fail to see the downside to that.

Quoted 2 lines

2) A process cannot initiate the request to dump itself, say from a signal handler. (since fork() is not signal safe)

This should be possible using libgdb. Let's say forking while in a SIGSEGV handler and using the libgdb API to do the dump.

Quoted 9 lines
Quoted 2 lines

Thats exactly the problem. forking within a sighandler is not safe. You could possibly deadlock with glibc locks.

Ondrej,

What are your thoughts about this ?

Thanks
Suzuki

Hi Suzuki,

from the LKML mailing list, I can see that the biggest criticism/confusion related to gencore comes from your necessity claims around the daemon part.

I'm not entirely sure what kind of programs is gencore going to be most used/useful for.. but isn't the signalfd API solving the problem of async-signal safety? Using it, you should be able to catch the signal, safely fork
and happily exec gencore. No need for any other daemon running.

Please correct me, if I'm mistaken.

Thanks,
Ondrej

#7 lore raw
Quoted 99 lines · Ondrej Oprala
Quoted 98 lines · Suzuki K. Poulose
Quoted 85 lines · Suzuki K. Poulose
Quoted 84 lines · Ondrej Oprala
Quoted 83 lines · Suzuki K. Poulose
Quoted 70 lines · Ondrej Oprala
Quoted 56 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of an application without
disrupting its process. The dump is collected non-disruptively and this tool currently supports
s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore"
which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to
be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This
is implemented through a daemon which listens on a UNIX Filesocket for
such requests. The daemon is started
immediately post installation. The program which requires the dump makes use of the gencore() API and provides
the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */
lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the
ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel
community. The patches have been posted at:https://lkml.org/lkml/2014/3/20/138

Quoted 13 lines

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe
util-linux" in
the body of a message [email protected]
More majordomo info athttp://vger.kernel.org/majordomo-info.html
Quoted 9 lines

Interesting,
but how is this different from attaching to a process with GDB and using
the gcore command? Or to automate it more, using the gcore script that
comes with GDB?
Cheers,
Ondrej

There are two major issues with that.

1) GDB uses PTRACE_ATTACH and hence the process gets a SIGSTOP.

Quoted 8 lines
Quoted 6 lines

I fail to see the downside to that.

Quoted 2 lines

2) A process cannot initiate the request to dump itself, say from a signal handler. (since fork() is not signal safe)

This should be possible using libgdb. Let's say forking while in a SIGSEGV
handler and using the libgdb API to do the dump.

Thats exactly the problem. forking within a sighandler is not safe. You could possibly deadlock with glibc locks.

Ondrej,

What are your thoughts about this ?

Quoted 7 lines
Quoted 3 lines

Thanks
Suzuki

Hi Suzuki,

from the LKML mailing list, I can see that the biggest criticism/confusion related to gencore comes from your necessity claims around the daemon part.

The daemon part was a shared philosophy from the CRIU project. There is no other reliable way of doing a self dump.

Quoted 3 lines

I'm not entirely sure what kind of programs is gencore going to be most used/useful for..

This can be used by huge applications, like, JAVA RUNTIME, to trigger a dump when it detects some issues, without actually bringing down the workload.

Quoted 3 lines

but isn't the signalfd API solving the problem of async-signal safety? Using it, you should be able to catch the signal, safely fork and happily exec gencore.

This imposes a lot of changes in the applications that may want to use the API and is prone to errors in attaining the same.

Quoted 2 lines

No need for any other daemon running.

The daemon doesn't add much overhead. With systemd, you could make use of the socket option to optimize the triggering of the gencore.

Btw, here is the link to the discussion about fork async-signal safety.
https://sourceware.org/bugzilla/show_bug.cgi?id=4737#c12

Quoted 5 lines

Please correct me, if I'm mistaken.

Thanks,
Ondrej

#8 lore raw
Quoted 103 lines · Suzuki K. Poulose
Quoted 102 lines · Ondrej Oprala
Quoted 101 lines · Suzuki K. Poulose
Quoted 85 lines · Suzuki K. Poulose
Quoted 84 lines · Ondrej Oprala
Quoted 83 lines · Suzuki K. Poulose
Quoted 70 lines · Ondrej Oprala
Quoted 56 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of an application without
disrupting its process. The dump is collected non-disruptively and this tool currently supports
s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also contains a library "libgencore"
which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along with name of the core-file to
be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This
is implemented through a daemon which listens on a UNIX Filesocket for
such requests. The daemon is started
immediately post installation. The program which requires the dump makes use of the gencore() API and provides
the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */
lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the
ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel
community. The patches have been posted at:https://lkml.org/lkml/2014/3/20/138

Quoted 13 lines

Do you think this can be part of the util-linux bundle? We can tweak it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe
util-linux" in
the body of a [email protected]
More majordomo info athttp://vger.kernel.org/majordomo-info.html
Quoted 9 lines

Interesting,
but how is this different from attaching to a process with GDB and using
the gcore command? Or to automate it more, using the gcore script that
comes with GDB?
Cheers,
Ondrej

There are two major issues with that.

1) GDB uses PTRACE_ATTACH and hence the process gets a SIGSTOP.

Quoted 8 lines
Quoted 6 lines

I fail to see the downside to that.

Quoted 2 lines

2) A process cannot initiate the request to dump itself, say from a signal handler. (since fork() is not signal safe)

This should be possible using libgdb. Let's say forking while in a SIGSEGV
handler and using the libgdb API to do the dump.

Thats exactly the problem. forking within a sighandler is not safe. You could possibly deadlock with glibc locks.

Ondrej,

What are your thoughts about this ?

Thanks
Suzuki

Quoted 6 lines
Quoted 4 lines

Hi Suzuki,

from the LKML mailing list, I can see that the biggest criticism/confusion related to gencore comes from your necessity claims around the daemon part.

The daemon part was a shared philosophy from the CRIU project. There is no other reliable way of doing a self dump.

Yes, I think that you explained the problem with self-ptrace clearly enough on the LKML.

Quoted 5 lines
Quoted 2 lines

I'm not entirely sure what kind of programs is gencore going to be most used/useful for..

This can be used by huge applications, like, JAVA RUNTIME, to trigger a dump when it detects some issues, without actually bringing down the workload.

Well, on 64-bit archs, huge programs may eat up terabytes of virtual memory, so normal dumps are sometimes close to impossible (though I'd really like to stress-test gdb with a massive 1TB coredump). Do you somehow get the process' VM size before dumping?
To limit the mappings to be dumped, for example...

Quoted 5 lines
Quoted 3 lines

but isn't the signalfd API solving the problem of async-signal safety? Using it, you should be able to catch the signal, safely fork and happily exec gencore.

This imposes a lot of changes in the applications that may want to use the API and is prone to errors in attaining the same.

But see, now we've moved from "CAN'T be done in any other way" to "CAN be done in other ways, although it might be non-trivial for some projects". I'm not saying the daemon doesn't have its usecases. I'm only trying to point out here, that there indeed ARE other ways.

Quoted 4 lines
Quoted 2 lines

No need for any other daemon running.

The daemon doesn't add much overhead. With systemd, you could make use of the socket option to optimize the triggering of the gencore.

I still haven't had time to look at the code itself. Does the daemon have to be running if I want to use the signalfd + fork + exec(gencore) approach mentioned above?

Quoted 2 lines

Btw, here is the link to the discussion about fork async-signal safety.
https://sourceware.org/bugzilla/show_bug.cgi?id=4737#c12

Yep, I've read that.

Quoted 5 lines
Quoted 5 lines

Please correct me, if I'm mistaken.

Thanks,
Ondrej

Cheers,
Ondrej

#9 lore raw
Quoted 139 lines · Ondrej Oprala
Quoted 108 lines · Suzuki K. Poulose
Quoted 107 lines · Ondrej Oprala
Quoted 106 lines · Suzuki K. Poulose
Quoted 90 lines · Suzuki K. Poulose
Quoted 89 lines · Ondrej Oprala
Quoted 84 lines · Suzuki K. Poulose
Quoted 74 lines · Ondrej Oprala
Quoted 59 lines · Janani Venkataraman

Hi,

We have developed a tool called "gencore" which captures the core of
an application without
disrupting its process. The dump is collected non-disruptively and this tool currently supports
s390, x86 and power systems.

THE TOOL:

The tool can perform non-disruptive third party dumps. The tool also
contains a library "libgencore"
which helps applicationsto trigger self dumps.

The tool can perform:

1) Third party dump: The pid of the process to dumped is given along
with name of the core-file to
be created.

eg.

[janani@localhost]:gencore 6616 core.test

2) Self dump: The programs can request a self-dump using gencore() API, provided throughlibgencore. This
is implemented through a daemon which listens on a UNIX Filesocket for
such requests. The daemon is started
immediately post installation. The program which requires the dump makes use of the gencore() API and provides
the name of the core-file as a parameter.

eg.

/* Opening the library, in this case the library is present in the /usr/lib64 */
lib = dlopen("libgencore.so", RTLD_LAZY);

gencore = dlsym(lib, "gencore");

Call the API:
gencore("/home/janani/core_test").

BASIC IDEA:

The basic idea is that the threads of the process are held using ptrace calls and the dump is generated in the
ELF format using the /proc/pid filesystem.

PATCH SET:
We have designed this tool based on the discussions with linux kernel
community. The patches have been posted at:https://lkml.org/lkml/2014/3/20/138

Quoted 14 lines

Do you think this can be part of the util-linux bundle? We can tweak
it to make it work as a package in util-linux.

Let us know your reviews and comments.

Thanks.
Janani

Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe
util-linux" in
the body of a [email protected]
More majordomo info athttp://vger.kernel.org/majordomo-info.html
Quoted 9 lines

Interesting,
but how is this different from attaching to a process with GDB and using
the gcore command? Or to automate it more, using the gcore script that
comes with GDB?
Cheers,
Ondrej

Quoted 3 lines

There are two major issues with that.

1) GDB uses PTRACE_ATTACH and hence the process gets a SIGSTOP.

I fail to see the downside to that.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

2) A process cannot initiate the request to dump itself, say from a signal handler. (since fork() is not signal safe)

This should be possible using libgdb. Let's say forking while in a SIGSEGV
handler and using the libgdb API to do the dump.

Thats exactly the problem. forking within a sighandler is not safe. You
could possibly deadlock with glibc locks.

Ondrej,

What are your thoughts about this ?

Thanks
Suzuki

Quoted 8 lines
Quoted 6 lines

Hi Suzuki,

from the LKML mailing list, I can see that the biggest criticism/confusion
related to gencore comes from your necessity claims around the daemon part.

The daemon part was a shared philosophy from the CRIU project. There is no other reliable way of doing a self dump.

Yes, I think that you explained the problem with self-ptrace clearly enough on the LKML.

Quoted 5 lines
Quoted 2 lines

I'm not entirely sure what kind of programs is gencore going to be most used/useful for..

This can be used by huge applications, like, JAVA RUNTIME, to trigger a dump when it detects some issues, without actually bringing down the workload.

Well, on 64-bit archs, huge programs may eat up terabytes of virtual memory, so normal dumps are sometimes close to impossible (though I'd really like to stress-test gdb with a massive 1TB coredump). Do you somehow get the process' VM size before dumping?
To limit the mappings to be dumped, for example...

Quoted 5 lines
Quoted 3 lines

but isn't the signalfd API solving the problem of async-signal safety? Using it, you should be able to catch the signal, safely fork and happily exec gencore.

This imposes a lot of changes in the applications that may want to use the API and is prone to errors in attaining the same.

But see, now we've moved from "CAN'T be done in any other way" to "CAN be done in other ways, although it might be non-trivial for some projects". I'm not saying the daemon doesn't have its usecases. I'm only trying to point out here, that there indeed ARE other ways.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

No need for any other daemon running.

The daemon doesn't add much overhead. With systemd, you could make use of the socket option to optimize the triggering of the gencore.

I still haven't had time to look at the code itself. Does the daemon have to be running if I want to use the signalfd + fork + exec(gencore) approach
mentioned above?

Sorry, this one was lost in other emails.
No we don't need a daemon if you can reliably invoke gencore

Cheers
Suzuki