autokdump - automated kdump testsuite
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]
I plan to release an automated kdump testsuite that will be focus on testing kernel and the crash utility. It should work for all major distros since it will use none of distro-specific stuff, and also support different arches including x86, ARM, PPC64 and s390x.
It does the following:
1) check if there is a memory reserved for kdump. If not, reserve the memory and reboot the system.
2) once the system is back, load kexec on panic and prepare a separate initramfs that including needed modules to load a local filesystem and necessary utilities in order to analyse /proc/vmcore in the 2nd kernel.
3) trigger the system crash using methods like sysrq-c, NMI, and panic_on_hung_task etc.
4) in the 2nd kernel, mount a filesystem and use the crash utility to analyse /proc/vmcore. Then, gather the analyse logs, serial console output, dmesg etc into the filesystem.
5) reboot back into the 1st kernel.
implementation:
It will setup a daemon to handle reboots.
plan:
I might also to test the makedumpfile all together later.
CAI Qian
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 1 line · CAI Qian
I plan to release an automated kdump testsuite that will be
So will this be a standalone test suit? Can it be merged with something already existing say, LTP.
Quoted 11 lines
focus on testing kernel and the crash utility. It should work for all major distros since it will use none of distro-specific stuff, and also support different arches including x86, ARM, PPC64 and s390x.
It does the following:
1) check if there is a memory reserved for kdump. If not, reserve the memory and reboot the system.
2) once the system is back, load kexec on panic and prepare a separate initramfs that including needed modules to load a local filesystem and necessary utilities
So you will write logic to prepare custom initramfs or will rely on dracut or some other utility for that.
Quoted 6 lines
in order to analyse
/proc/vmcorein the 2nd kernel.
3) trigger the system crash using methods like sysrq-c, NMI, andpanic_on_hung_tasketc.
4) in the 2nd kernel, mount a filesystem and use the crash utility to analyse/proc/vmcore. Then, gather the analyse logs, serial console output, dmesg etc into the filesystem.
Why not save core and boot back in first kernel and then analyze.
Trying to work directly with /proc/vmcore does not test makedumfile which everybody uses. Also it will require more memory to be reserved and packing crash and debug vmlinux into initramfs.
I think being able to test makedumpfile also is the key here.
Thanks
Vivek
Quoted 8 lines
5) reboot back into the 1st kernel.
implementation:
It will setup a daemon to handle reboots.plan:
I might also to test the makedumpfile all together later.
CAI Qian
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 5 lines · Vivek Goyal
Quoted 1 line · CAI Qian
I plan to release an automated kdump testsuite that will be
So will this be a standalone test suit? Can it be merged with something already existing say, LTP.
Also how about calling it just kdumptest. autokdump feels more like as if there have been improvements to kdump mechanism and there is improved method named "autokdump".
kdumptest is more intuitive and it being "auto" is more or less implied.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
----- Original Message -----
Quoted 8 lines
From: "Vivek Goyal" <[email protected]>
To: "CAI Qian" <[email protected]>
Cc: "linux-kernel" <[email protected]>, "ltp-list" <[email protected]>, "crash-utility" <[email protected]>, "kexec" <[email protected]>, "kexec kdump redhat mailing list" <[email protected]>
Sent: Friday, September 19, 2014 9:22:36 PM
Subject: Re: [RFC] autokdump - automated kdump testsuite
Quoted 5 lines
Quoted 1 line · CAI Qian
I plan to release an automated kdump testsuite that will be
So will this be a standalone test suit? Can it be merged with something already existing say, LTP.
Yes, it is likely to be standalone. It won't make use of the LTP API, and the LTP kdump test suite is outdated, so there is no benefit to continue working over there.
Quoted 13 lines
Quoted 6 lines
focus on testing kernel and the crash utility. It should work for all major distros since it will use none of distro-specific stuff, and also support different arches including x86, ARM, PPC64 and s390x.
Quoted 6 lines
It does the following:
1) check if there is a memory reserved for kdump. If not, reserve the memory and reboot the system.
2) once the system is back, load kexec on panic and prepare a separate initramfs that including needed modules to load a local filesystem and necessary utilities
Quoted 2 lines
So you will write logic to prepare custom initramfs or will rely on dracut or some other utility for that.
I'll probably prepare custom initramfs for the sake of simplicity.
Quoted 8 lines
Quoted 7 lines
in order to analyse
/proc/vmcorein the 2nd kernel.
3) trigger the system crash using methods like sysrq-c, NMI, andpanic_on_hung_tasketc.
4) in the 2nd kernel, mount a filesystem and use the crash utility to analyse/proc/vmcore. Then, gather the analyse logs, serial console output, dmesg etc into the filesystem.
Quoted 5 lines
Why not save core and boot back in first kernel and then analyze.
Trying to work directly with
/proc/vmcoredoes not test makedumfile which everybody uses. Also it will require more memory to be reserved and packing crash and debug vmlinux into initramfs.
The additional memory for vmlinux and the crash utility is predictable and manageable, so it can just ask 256M memory reserved before running the program. On the other hand, it is not usually feasible to ask the systems under testing has enough available disk spaces bigger than the memory size.
CAI Qian
Quoted 14 lines
I think being able to test makedumpfile also is the key here.
Thanks
VivekQuoted 8 lines
5) reboot back into the 1st kernel.
implementation:
It will setup a daemon to handle reboots.plan:
I might also to test the makedumpfile all together later.
CAI Qian
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 11 lines · CAI Qian
----- Original Message -----
Quoted 8 lines
From: "Vivek Goyal" <[email protected]>
To: "CAI Qian" <[email protected]>
Cc: "linux-kernel" <[email protected]>, "ltp-list" <[email protected]>, "crash-utility" <[email protected]>, "kexec" <[email protected]>, "kexec kdump redhat mailing list" <[email protected]>
Sent: Friday, September 19, 2014 9:22:36 PM
Subject: Re: [RFC] autokdump - automated kdump testsuite
Quoted 8 lines
Quoted 5 lines · CAI Qian
Quoted 1 line
I plan to release an automated kdump testsuite that will be
So will this be a standalone test suit? Can it be merged with something already existing say, LTP.
Yes, it is likely to be standalone. It won't make use of the LTP API, and the LTP kdump test suite is outdated, so there is no benefit to continue working over there.
So why make it standalone and not replace the old LTP kdump test suite with this new one?
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
focus on testing kernel and the crash utility. It should work for all major distros since it will use none of distro-specific stuff, and also support different arches including x86, ARM, PPC64 and s390x.
It does the following:
1) check if there is a memory reserved for kdump. If not, reserve the memory and reboot the system.
2) once the system is back, load kexec on panic and prepare a separate initramfs that including needed modules to load a local filesystem and necessary utilities
Quoted 3 lines
Quoted 2 lines
So you will write logic to prepare custom initramfs or will rely on dracut or some other utility for that.
I'll probably prepare custom initramfs for the sake of simplicity.
Well, preparing custom initramfs will become very tricky. We used to do that and finally we switched to dracut.
Why not simply let the respective service on the host do this job and test only makes sure that kdump service is running. It feels little out of place that a test is generating custom initramfs.
Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
in order to analyse
/proc/vmcorein the 2nd kernel.
3) trigger the system crash using methods like sysrq-c, NMI, andpanic_on_hung_tasketc.
4) in the 2nd kernel, mount a filesystem and use the crash utility to analyse/proc/vmcore. Then, gather the analyse logs, serial console output, dmesg etc into the filesystem.Why not save core and boot back in first kernel and then analyze.
Quoted 8 lines
Quoted 3 lines
Trying to work directly with
/proc/vmcoredoes not test makedumfile which everybody uses. Also it will require more memory to be reserved and packing crash and debug vmlinux into initramfs.The additional memory for vmlinux and the crash utility is predictable and manageable, so it can just ask 256M memory reserved before running the program. On the other hand, it is not usually feasible to ask the systems under testing has enough available disk spaces bigger than the memory size.
makedumpfile will reduce the vmcore file size to few hundreds of mega bytes on most of the systems. Especially, this is just a test, so system will be lightly loaded and vmcore will be small after filtering.
If there is not enough space, test fails, period. I don't think there is any need to try to circumvent that and try to run crash in initramfs. And in the process we don't test makedumpfile which is very imporatnt component of this whole process.
IMHO, just rely on "systemctl start kdump" to generate and load custom initramfs and save filtered vmcore to root fs by default and alanyze vmcore post reboot. That will keep things simple.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
----- Original Message -----
Quoted 25 lines
From: "Vivek Goyal" <[email protected]>
To: "CAI Qian" <[email protected]>
Cc: "ltp-list" sourceforge.net>, "kexec kdump redhat mailing list" <[email protected]>, "kexec" <[email protected]>, "linux-kernel" vger.kernel.org>, "crash-utility" redhat.com>
Sent: Monday, September 22, 2014 10:47:13 PM
Subject: Re: [RFC] autokdump - automated kdump testsuiteQuoted 16 lines · CAI Qian
----- Original Message -----
Quoted 13 lines
From: "Vivek Goyal" <[email protected]>
To: "CAI Qian" <[email protected]>
Cc: "linux-kernel" vger.kernel.org>, "ltp-list" sourceforge.net>, "crash-utility"
redhat.com>, "kexec" <[email protected]>, "kexec kdump redhat mailing list" <[email protected]>
Sent: Friday, September 19, 2014 9:22:36 PM
Subject: Re: [RFC] autokdump - automated kdump testsuiteQuoted 1 line · CAI Qian
I plan to release an automated kdump testsuite that will be
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
So will this be a standalone test suit? Can it be merged with something already existing say, LTP.
Yes, it is likely to be standalone. It won't make use of the LTP API, and the LTP kdump test suite is outdated, so there is no benefit to continue working over there.
So why make it standalone and not replace the old LTP kdump test suite with this new one?
It will be totally a rewrite from scratch, and will have no direct relationship with the rest of the LTP.
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 7 lines
focus on testing kernel and the crash utility. It should work for all major distros since it will use none of distro-specific stuff, and also support different arches including x86, ARM, PPC64 and s390x.
Quoted 6 lines
It does the following:
1) check if there is a memory reserved for kdump. If not, reserve the memory and reboot the system.
2) once the system is back, load kexec on panic and prepare a separate initramfs that including needed modules to load a local filesystem and necessary utilitiesSo you will write logic to prepare custom initramfs or will rely on dracut or some other utility for that.
Quoted 8 lines
Quoted 1 line
I'll probably prepare custom initramfs for the sake of simplicity.
Well, preparing custom initramfs will become very tricky. We used to do that and finally we switched to dracut.
Why not simply let the respective service on the host do this job and test only makes sure that kdump service is running. It feels little out of place that a test is generating custom initramfs.
Because not every distro will have a kdump service like Fedora.
Quoted 20 lines
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines
in order to analyse
/proc/vmcorein the 2nd kernel.
3) trigger the system crash using methods like sysrq-c, NMI, andpanic_on_hung_tasketc.
4) in the 2nd kernel, mount a filesystem and use the crash utility to analyse/proc/vmcore. Then, gather the analyse logs, serial console output, dmesg etc into the filesystem.Why not save core and boot back in first kernel and then analyze.
Quoted 8 lines
Quoted 3 lines
Trying to work directly with
/proc/vmcoredoes not test makedumfile which everybody uses. Also it will require more memory to be reserved and packing crash and debug vmlinux into initramfs.The additional memory for vmlinux and the crash utility is predictable and manageable, so it can just ask 256M memory reserved before running the program. On the other hand, it is not usually feasible to ask the systems under testing has enough available disk spaces bigger than the memory size.
Quoted 3 lines
makedumpfile will reduce the vmcore file size to few hundreds of mega bytes on most of the systems. Especially, this is just a test, so system will be lightly loaded and vmcore will be small after filtering.
It probably actually have test cases to heavily loaded the memory before dumping.
Quoted 5 lines
If there is not enough space, test fails, period. I don't think there is any need to try to circumvent that and try to run crash in initramfs. And in the process we don't test makedumpfile which is very imporatnt component of this whole process.
Test makedumpfile is in plan. Tests failed because of lack of disk space is a testsuite design problem, and especially problematic on those large memory systems that we had seen more and more those days.
Quoted 4 lines
IMHO, just rely on "systemctl start kdump" to generate and load custom initramfs and save filtered vmcore to root fs by default and alanyze vmcore post reboot. That will keep things simple.
Again, not every major distro has that.
CAI Qian
Quoted 9 lines
Thanks
Vivek_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
On Mon, Sep 22, 2014 at 10:53:55PM -0400, CAI Qian wrote:
[..]
Quoted 4 lines
Quoted 3 lines
Why not simply let the respective service on the host do this job and test only makes sure that kdump service is running. It feels little out of place that a test is generating custom initramfs.
Because not every distro will have a kdump service like Fedora.
So which distro does not have a service? Do we know this or we are assuming that distributions don't have a service to load/unload kdump kernel.
[..]
Quoted 5 lines
Quoted 3 lines
makedumpfile will reduce the vmcore file size to few hundreds of mega bytes on most of the systems. Especially, this is just a test, so system will be lightly loaded and vmcore will be small after filtering.
It probably actually have test cases to heavily loaded the memory before dumping.
Your original proposal does not take care of this case either. A test case could be heavily used memory and if user does not have sufficient memory to save core, so be it. That test will fail.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hi,
Quoted 5 lines · CAI Qian
I plan to release an automated kdump testsuite that will be focus on testing kernel and the crash utility. It should work for all major distros since it will use none of distro-specific stuff, and also support different arches including x86, ARM, PPC64 and s390x.
It's a good idea to create a kdump test project. Maybe you can consider to test kexec as well.
Quoted 15 lines
It does the following:
1) check if there is a memory reserved for kdump. If not, reserve the memory and reboot the system.
2) once the system is back, load kexec on panic and prepare a separate initramfs that including needed modules to load a local filesystem and necessary utilities in order to analyse/proc/vmcorein the 2nd kernel.
3) trigger the system crash using methods like sysrq-c, NMI, andpanic_on_hung_tasketc.
4) in the 2nd kernel, mount a filesystem and use the crash utility to analyse/proc/vmcore. Then, gather the analyse logs, serial console output, dmesg etc into the filesystem.
5) reboot back into the 1st kernel.
Quoted 5 lines
implementation:
It will setup a daemon to handle reboots.plan:
I might also to test the makedumpfile all together later.
For makedumpfile, IMO you can add several test cases and collect the dump time, vmcore size data as test results besides of the dump status.
Thanks
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 20 lines · CAI Qian
I plan to release an automated kdump testsuite that will be focus on testing kernel and the crash utility. It should work for all major distros since it will use none of distro-specific stuff, and also support different arches including x86, ARM, PPC64 and s390x.
It does the following:
1) check if there is a memory reserved for kdump. If not, reserve the memory and reboot the system.
2) once the system is back, load kexec on panic and prepare a separate initramfs that including needed modules to load a local filesystem and necessary utilities in order to analyse/proc/vmcorein the 2nd kernel.
3) trigger the system crash using methods like sysrq-c, NMI, andpanic_on_hung_tasketc.
4) in the 2nd kernel, mount a filesystem and use the crash utility to analyse/proc/vmcore. Then, gather the analyse logs, serial console output, dmesg etc into the filesystem.
5) reboot back into the 1st kernel.
Quoted 7 lines
implementation:
It will setup a daemon to handle reboots.plan:
I might also to test the makedumpfile all together later.
CAI Qian
Hi, Cai
I noticed you cced redhat internal lists, you'd better to remove them from cc list so that people do not get confused by the auto reply.
Thanks
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec