makedumpfile header to show LZO/snappy/zlib
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]
RFC PATCH 1/1 makedumpfile header to show LZO/snappy/zlib
With the advent of LZO and snappy compression support, it would be helpful if it could be readily determined which type of compression was used to create the dumpfile.
The zlib/LZO/snappy indicator is only found down in the bowels of each page_desc structure:
/* page flags */
#define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib */
#define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo */
#define DUMP_DH_COMPRESSED_SNAPPY 0x4
/* paged is compressed with snappy */ /* descriptor of each page for vmcore */
typedef struct page_desc {
off_t offset; /* the offset of the page data*/
unsigned int size; /* the size of this dump page */
unsigned int flags; /* flags */
unsigned long long page_flags; /* page flags */ } page_desc_t;
The compressed kdump's header does contain a "status" flag:
struct disk_dump_header {
char signature[SIG_LEN]; /* = "KDUMP " */
int header_version; /* Dump header version */
struct new_utsname utsname; /* copy of system_utsname */
struct timeval timestamp; /* Time stamp */
unsigned int status; /* Above flags */...
which has these 3 flags:
#define DUMP_HEADER_COMPLETED 0 #define DUMP_HEADER_INCOMPLETED 1 #define DUMP_HEADER_COMPRESSED 8
But they are all remnants of the old "diskdump" facility, and none of them are ever used by makedumpfile.
Any chance that the LZO/snappy/zlib method could also be put up in the disk_dump_header?
Dave
makedumpfile.c renamed from makedumpfile.c.orig +10 -0
@@ -5162,6 +5162,16 @@ write_kdump_header(void) dh->bitmap_blocks =divideup (info->len_bitmap, dh->block_size);memcpy (&dh->timestamp, &info->timestamp,sizeof (dh->timestamp));memcpy (&dh->utsname, &info->system_utsname,sizeof (dh->utsname)); +if (info->flag_compress &DUMP_DH_COMPRESSED_ZLIB ) + dh->status |=DUMP_DH_COMPRESSED_ZLIB ; +#ifdef USELZO + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) + dh->status |= DUMP_DH_COMPRESSED_LZO) +#endif +#ifdef USESNAPPY + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) +#endif size =sizeof (struct disk_dump_header );if (!write_buffer (info->fd_dumpfile,0 , dh, size, info->name_dumpfile))
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello Dave,
Quoted 52 lines · Dave Anderson
With the advent of LZO and snappy compression support, it would be helpful if it could be readily determined which type of compression was used to create the dumpfile.
The zlib/LZO/snappy indicator is only found down in the bowels of each
page_descstructure:/* page flags */ #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib */ #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo */ #define DUMP_DH_COMPRESSED_SNAPPY 0x4 /* paged is compressed with snappy *//* descriptor of each page for vmcore */
typedef structpage_desc{off_t offset; /* the offset of the page data*/ unsigned int size; /* the size of this dump page */ unsigned int flags; /* flags */ unsigned long long page_flags; /* page flags */}
page_desc_t;The compressed kdump's header does contain a "status" flag:
struct
disk_dump_header{char signature[SIG_LEN]; /* = "KDUMP " */ int header_version; /* Dump header version */ struct new_utsname utsname; /* copy of system_utsname */ struct timeval timestamp; /* Time stamp */ unsigned int status; /* Above flags */...
which has these 3 flags:
#define DUMP_HEADER_COMPLETED 0 #define DUMP_HEADER_INCOMPLETED 1 #define DUMP_HEADER_COMPRESSED 8But they are all remnants of the old "diskdump" facility, and none of them are ever used by makedumpfile.
Any chance that the LZO/snappy/zlib method could also be put up in the
disk_dump_header?Dave
makedumpfile.c renamed from makedumpfile.c.orig +0
-0@@ -5162,6 +5162,16 @@ write_kdump_header(void) dh->bitmap_blocks =divideup (info->len_bitmap, dh->block_size);memcpy (&dh->timestamp, &info->timestamp,sizeof (dh->timestamp));memcpy (&dh->utsname, &info->system_utsname,sizeof (dh->utsname));
makedumpfile.c write_kdump_header(void) · line 5174
Quoted 10 lines
makedumpfile.c renamed from makedumpfile.c.orig +10
-0+if (info->flag_compress &DUMP_DH_COMPRESSED_ZLIB ) + dh->status |=DUMP_DH_COMPRESSED_ZLIB ; +#ifdef USELZO + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) + dh->status |= DUMP_DH_COMPRESSED_LZO) +#endif +#ifdef USESNAPPY + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) +#endif
dh->status will not guarantee that *all pages* are compressed by the check below:
Quoted 7 lines
makedumpfile.c renamed from makedumpfile.c.orig +0
-0#ifdef USELZO }else if (info->flag_lzo_support && (info->flag_compress & DUMP_DH_COMPRESSED_LZO)&& ((size_out = info->page_size), lzo1x_1_compress(buf, info->page_size, buf_out, &size_out, wrkmem) == LZO_E_OK) && (size_out < info->page_size)) {
^^^^^^^^^^^^^^^^^^^^^^^^^^^
When there is no effect of compression, the page will not be compressed.
Quoted 4 lines
pd.flags = DUMP_DH_COMPRESSED_LZO; pd.size = size_out; memcpy(buf, buf_out, pd.size);#endif
If we are on the same page, I'll accept your patch.
Thanks
Atsushi Kumagai
Quoted 3 lines
size = sizeof(struct disk_dump_header);
if (!write_buffer(info->fd_dumpfile, 0, dh, size, info->name_dumpfile))_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
----- Original Message -----
Quoted 88 lines
Hello Dave,
Quoted 60 lines · Dave Anderson
With the advent of LZO and snappy compression support, it would be helpful if it could be readily determined which type of compression was used to create the dumpfile.
The zlib/LZO/snappy indicator is only found down in the bowels of each
page_descstructure:/* page flags */ #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib */ #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo */ #define DUMP_DH_COMPRESSED_SNAPPY 0x4 /* paged is compressed with snappy *//* descriptor of each page for vmcore */
typedef structpage_desc{
off_toffset; /* the offset of the
page data*/
unsigned int size; /* the size of this dump
page */
unsigned int flags; /* flags */
unsigned long longpage_flags; /* page flags */
}page_desc_t;The compressed kdump's header does contain a "status" flag:
struct
disk_dump_header{
char signature[SIG_LEN]; /* = "KDUMP "
*/
intheader_version; /* Dump header version
*/
structnew_utsnameutsname; /* copy of
system_utsname*/
struct timeval timestamp; /* Time stamp */
unsigned int status; /* Above flags */
...which has these 3 flags:
#define DUMP_HEADER_COMPLETED 0 #define DUMP_HEADER_INCOMPLETED 1 #define DUMP_HEADER_COMPRESSED 8But they are all remnants of the old "diskdump" facility, and none of them are ever used by makedumpfile.
Any chance that the LZO/snappy/zlib method could also be put up in the
disk_dump_header?Dave
makedumpfile.c renamed from makedumpfile.c.orig +0
-0@@ -5162,6 +5162,16 @@ write_kdump_header(void) dh->bitmap_blocks =divideup (info->len_bitmap, dh->block_size);memcpy (&dh->timestamp, &info->timestamp,sizeof (dh->timestamp));memcpy (&dh->utsname, &info->system_utsname,sizeof (dh->utsname));makedumpfile.c write_kdump_header(void) · line 5174
Quoted 10 lines
makedumpfile.c renamed from makedumpfile.c.orig +10
-0+if (info->flag_compress &DUMP_DH_COMPRESSED_ZLIB ) + dh->status |=DUMP_DH_COMPRESSED_ZLIB ; +#ifdef USELZO + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) + dh->status |= DUMP_DH_COMPRESSED_LZO) +#endif +#ifdef USESNAPPY + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) +#endif
dh->statuswill not guarantee that *all pages* are compressed by the check below:Quoted 10 lines
makedumpfile.c renamed from makedumpfile.c.orig +0
-0#ifdef USELZO } else if (info->flag_lzo_support && (info->flag_compress &DUMP_DH_COMPRESSED_LZO) && ((size_out = info->page_size), lzo1x_1_compress(buf, info->page_size, buf_out, &size_out, wrkmem) == LZO_E_OK) && (size_out < info->page_size)) {^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 8 lines
When there is no effect of compression, the page will not be compressed.
Quoted 4 lines
pd.flags = DUMP_DH_COMPRESSED_LZO; pd.size = size_out; memcpy(buf, buf_out, pd.size);#endif
If we are on the same page, I'll accept your patch.
Right, I understand.
Thanks,
Dave
Quoted 10 lines
Thanks
Atsushi KumagaiQuoted 4 lines
size = sizeof(struct disk_dump_header); if (!write_buffer(info->fd_dumpfile, 0, dh, size, info->name_dumpfile))
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 93 lines · On Fri, 20 Sep 2013 09:04:40 -0400 (EDT) Dave Anderson <[email protected]>…
----- Original Message -----
Quoted 90 lines
Hello Dave,
Quoted 60 lines · Dave Anderson
With the advent of LZO and snappy compression support, it would be helpful if it could be readily determined which type of compression was used to create the dumpfile.
The zlib/LZO/snappy indicator is only found down in the bowels of each
page_descstructure:/* page flags */ #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib */ #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo */ #define DUMP_DH_COMPRESSED_SNAPPY 0x4 /* paged is compressed with snappy *//* descriptor of each page for vmcore */
typedef structpage_desc{
off_toffset; /* the offset of the
page data*/
unsigned int size; /* the size of this dump
page */
unsigned int flags; /* flags */
unsigned long longpage_flags; /* page flags */
}page_desc_t;The compressed kdump's header does contain a "status" flag:
struct
disk_dump_header{
char signature[SIG_LEN]; /* = "KDUMP "
*/
intheader_version; /* Dump header version
*/
structnew_utsnameutsname; /* copy of
system_utsname*/
struct timeval timestamp; /* Time stamp */
unsigned int status; /* Above flags */
...which has these 3 flags:
#define DUMP_HEADER_COMPLETED 0 #define DUMP_HEADER_INCOMPLETED 1 #define DUMP_HEADER_COMPRESSED 8But they are all remnants of the old "diskdump" facility, and none of them are ever used by makedumpfile.
Any chance that the LZO/snappy/zlib method could also be put up in the
disk_dump_header?Dave
makedumpfile.c renamed from makedumpfile.c.orig +0
-0@@ -5162,6 +5162,16 @@ write_kdump_header(void) dh->bitmap_blocks =divideup (info->len_bitmap, dh->block_size);memcpy (&dh->timestamp, &info->timestamp,sizeof (dh->timestamp));memcpy (&dh->utsname, &info->system_utsname,sizeof (dh->utsname));makedumpfile.c write_kdump_header(void) · line 5174
Quoted 10 lines
makedumpfile.c renamed from makedumpfile.c.orig +10
-0+if (info->flag_compress &DUMP_DH_COMPRESSED_ZLIB ) + dh->status |=DUMP_DH_COMPRESSED_ZLIB ; +#ifdef USELZO + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) + dh->status |= DUMP_DH_COMPRESSED_LZO) +#endif +#ifdef USESNAPPY + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) +#endif
dh->statuswill not guarantee that *all pages* are compressed by the check below:Quoted 10 lines
makedumpfile.c renamed from makedumpfile.c.orig +0
-0#ifdef USELZO } else if (info->flag_lzo_support && (info->flag_compress &DUMP_DH_COMPRESSED_LZO) && ((size_out = info->page_size), lzo1x_1_compress(buf, info->page_size, buf_out, &size_out, wrkmem) == LZO_E_OK) && (size_out < info->page_size)) {^^^^^^^^^^^^^^^^^^^^^^^^^^^When there is no effect of compression, the page will not be compressed.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
pd.flags = DUMP_DH_COMPRESSED_LZO; pd.size = size_out; memcpy(buf, buf_out, pd.size);#endif
If we are on the same page, I'll accept your patch.
Right, I understand.
I think the primary intention is to let crash determine early if it is able to open a given dump or not. And that's a good idea!
From a semantic point of view, the status field in the header should be a logical OR of all compression methods used in the coredump. AFAICS, Dave's patch would actually work for the current implementation.
Did I miss something?
Petr T
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
----- Original Message -----
Quoted 95 lines
Quoted 92 lines · On Fri, 20 Sep 2013 09:04:40 -0400 (EDT) Dave Anderson <[email protected]>…
----- Original Message -----
Quoted 89 lines
Hello Dave,
Quoted 54 lines · Dave Anderson
With the advent of LZO and snappy compression support, it would be helpful if it could be readily determined which type of compression was used to create the dumpfile.
The zlib/LZO/snappy indicator is only found down in the bowels of each
page_descstructure:/* page flags */ #define DUMP_DH_COMPRESSED_ZLIB 0x1 /* page is compressed with zlib */ #define DUMP_DH_COMPRESSED_LZO 0x2 /* paged is compressed with lzo */ #define DUMP_DH_COMPRESSED_SNAPPY 0x4 /* paged is compressed with snappy *//* descriptor of each page for vmcore */
typedef structpage_desc{off_t offset; /* the offset of the page data */ unsigned int size; /* the size of this dump page */ unsigned int flags; /* flags */ unsigned long long page_flags; /* page flags */}
page_desc_t;The compressed kdump's header does contain a "status" flag:
struct
disk_dump_header{char signature[SIG_LEN]; /* = "KDUMP " */ int header_version; /* Dump header version */ struct new_utsname utsname; /* copy of system_utsname */ struct timeval timestamp; /* Time stamp */ unsigned int status; /* Above flags */...
which has these 3 flags:
#define DUMP_HEADER_COMPLETED 0 #define DUMP_HEADER_INCOMPLETED 1 #define DUMP_HEADER_COMPRESSED 8But they are all remnants of the old "diskdump" facility, and none of them are ever used by makedumpfile.
Any chance that the LZO/snappy/zlib method could also be put up in the
disk_dump_header?Dave
makedumpfile.c renamed from makedumpfile.c.orig +0
-0@@ -5162,6 +5162,16 @@ write_kdump_header(void) dh->bitmap_blocks =divideup (info->len_bitmap, dh->block_size);memcpy (&dh->timestamp, &info->timestamp,sizeof (dh->timestamp));memcpy (&dh->utsname, &info->system_utsname,sizeof (dh->utsname));makedumpfile.c write_kdump_header(void) · line 5174
Quoted 10 lines
makedumpfile.c renamed from makedumpfile.c.orig +10
-0+if (info->flag_compress &DUMP_DH_COMPRESSED_ZLIB ) + dh->status |=DUMP_DH_COMPRESSED_ZLIB ; +#ifdef USELZO + else if (info->flag_compress & DUMP_DH_COMPRESSED_LZO) + dh->status |= DUMP_DH_COMPRESSED_LZO) +#endif +#ifdef USESNAPPY + else if (info->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) + dh->status |= DUMP_DH_COMPRESSED_SNAPPY) +#endif
dh->statuswill not guarantee that *all pages* are compressed by the check below:Quoted 10 lines
makedumpfile.c renamed from makedumpfile.c.orig +0
-0#ifdef USELZO } else if (info->flag_lzo_support && (info->flag_compress &DUMP_DH_COMPRESSED_LZO) && ((size_out = info->page_size), lzo1x_1_compress(buf, info->page_size, buf_out, &size_out, wrkmem) == LZO_E_OK) && (size_out < info->page_size)) {^^^^^^^^^^^^^^^^^^^^^^^^^^^When there is no effect of compression, the page will not be compressed.
Quoted 4 lines
pd.flags = DUMP_DH_COMPRESSED_LZO; pd.size = size_out; memcpy(buf, buf_out, pd.size);#endif
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
If we are on the same page, I'll accept your patch.
Right, I understand.
I think the primary intention is to let crash determine early if it is able to open a given dump or not. And that's a good idea!
The primary intention is informational only, either for "help -[nD]" or for the "crash -d# ..." display which dumps the header contents immediately
after reading it. As it is now, if crash is unable to support the compression
type because the relevant library was not built in, it will accept the vmcore, but will fail upon the first read attempt with a message "uncompress failed: no lzo compression support" (or snappy).
Quoted 7 lines
From a semantic point of view, the status field in the header should be a logical OR of all compression methods used in the coredump. AFAICS, Dave's patch would actually work for the current implementation.
Did I miss something?
Petr T
I don't think so -- Atsushi's point was simply that there is a good probability that there will be a mixture of compressed and uncompressed pages.
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec