makedumpfile security key filtering with eppic
Subsystems (3)
v2 cover makedumpfile security key filtering with eppic
makedumpfile security key filtering enhancement - Add Eppic language support (formerly known as SIAL) to specify rules to scrub data in a dumpfile. Eppic was previously part of crash source code repository.
The following series of patches enhance the makedumpfile to provide a more powerful way to specify rules and commands to traverse and erase complex data structures in a dump file by integrating Embeddable Pre-Processor and Interpreter for C (eppic).
Eppic is an interpreter that facilitates access to the symbol and type information stored in an executable image or a dump file. Eppic defines a language semantic which is similar to C. Eppic macros can be used to specify rules/commands to erase data in an image file. makedumpfile will interpret the rules/commands provided by eppic macros with the help of eppic library and will suitably erase the required data in a dump file. Eppic provides a lot of language constructs like conditional statements, logical and arithmetic operators, nested loops, functions, etc., to traverse nested lists and trees and conditionally erase data in the dump file, enabling users to literally erase any data in the dump file which is accessible through global symbols.
The series of patches integrates eppic with makdumpfile. These patches require eppic library libeppic.a and eppic_api.h header file. The libeppic.a library can be built from the eppic source code available at the following URL:
http://code.google.com/p/eppic/
TODO:
- Currently, works only for symbols in vmlinux, extend it to module symbols
- Functionality support:
- Implement the following callback functions.
- apialignment
- apigetenum
- apigetdefs
- Other functionalities specified in the code with TODO tag
- Support specifying eppic macros in makedumpfile.conf file
- Update erase info
Changelog from v1 to v2:
- Re-based to v1.5.0
- Introduced EPPIC=on in makefile, and hence eppic is now optional
- Incorporated review comments from Atsushi
- Minor formatting changes
Regards,
Aravinda
---
Aravinda Prasad (7):
Initialize and setup eppic
makedumpfile and eppic interface layer
Eppic call back functions to query a dump image Implement apigetctype call back function Implement apimember and apigetrtype call back functions Extend eppic built-in functions to include memset function Support fully typed symbol access mode
Makefile | 7 + dwarf_info.c | 367 +++++++++++++++++++++++++++++++++++++++++++ dwarf_info.h | 18 ++ erase_info.c | 83 +++++++++- erase_info.h | 5 + extension_eppic.c | 451 +++++++++++++++++++++++++++++++++++++++++++++++++++++ extension_eppic.h | 78 +++++++++ makedumpfile.c | 7 + makedumpfile.h | 6 + 9 files changed, 1018 insertions(+), 4 deletions(-) create mode 100644 extension_eppic.c create mode 100644 extension_eppic.h
Signature · 7 lines
Aravinda Prasad _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 1/7 Initialize and setup eppic
This patch contains routines which initialize eppic and register call back function which will be called whenever a new eppic macro is loaded using eppic_load() API. The registered call back function executes the eppic macro as soon as it is loaded.
The -ltinfo is included in LIBS in the Makefile, because eppic currently calls few functions in libtinfo and the compiler complains about it, if not included. I think the paths where libeppic calls libtinfo functions are not exercised by makedumpfile.
Including -ltinfo will increase the size of makedumpfile static version
TODO:
- Check out a way to get rid of -ltinfo
--- Makefile | 7 ++++ extension_eppic.c | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++ extension_eppic.h | 23 +++++++++++++++ 3 files changed, 114 insertions(+), 0 deletions(-) create mode 100644 extension_eppic.c create mode 100644 extension_eppic.h
Makefile +7 -0
@@ -55,6 +55,13 @@ LIBS := -llzo2 $(LIBS) CFLAGS += -DUSELZO endif +ifeq ($(EPPIC), on) +LIBS := -leppic -ltinfo $(LIBS) +CFLAGS += -DEPPIC +SRC_PART += extension_eppic.c +OBJ_PART += extension_eppic.o +endif + all: makedumpfile $(OBJ_PART): $(SRC_PART)
extension_eppic.c new file +84 -0
84 lines not on this page · raw message
extension_eppic.h new file +23 -0
@@ -0,0 +1,23 @@ +/* +* extension_eppic.h +* +* Created by: Aravinda Prasad <[email protected]> +* +* Copyright (C) 2012 IBM Corporation +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +*/ +#ifndef _EXTENSION_EPPIC_H +#define _EXTENSION_EPPIC_H + +#include "eppic_api.h" + +#endif /* _EXTENSION_EPPIC_H */
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v2 1/7] Initialize and setup eppic
On Thu, Nov 08, 2012 at 07:08:04PM +0530, Aravinda Prasad wrote:
[..]
Makefile LIBS := -llzo2 $(LIBS) · line 59
Quoted 10 lines
Makefile +2
-0@@ -55,6 +55,13 @@ LIBS := -llzo2 $(LIBS) CFLAGS += -DUSELZO endif +ifeq ($(EPPIC), on) +LIBS := -leppic -ltinfo $(LIBS)
So you also pull in libarary "tinfo". Does that 1MB number include this too? What's the total size bloat for makedumpfile once we compile with EPPIC on.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v2 1/7] Initialize and setup eppic
Quoted 11 lines · Vivek Goyal
On Thu, Nov 08, 2012 at 07:08:04PM +0530, Aravinda Prasad wrote:
[..]
Quoted 8 lines
Makefile +0
-0@@ -55,6 +55,13 @@ LIBS := -llzo2 $(LIBS) CFLAGS += -DUSELZO endif
Quoted 6 lines
Makefile LIBS := -llzo2 $(LIBS) · line 59
Quoted 2 lines
Makefile +2
-0+ifeq ($(EPPIC), on) +LIBS := -leppic -ltinfo $(LIBS)So you also pull in libarary "tinfo". Does that 1MB number include this too? What's the total size bloat for makedumpfile once we compile with EPPIC on.
Yes it includes. The actual size bloat for makedumpfile with EPPIC=on is around 850KB
Quoted 4 lines
Thanks
Vivek
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 2/7 makedumpfile and eppic interface layer
In reply to Aravinda PrasadThis patch extends the makedumpfile functionality to include eppic language support. The patch initializes eppic and then loads and executes the specified eppic macros.
This patch also includes "--eppic" option to makedumpfile command to specify the eppic macro or the directory containing eppic macros. The specified eppic macro will be executed to scrub the data in the dumpfile. In case of a directory, all the eppic macros inside the directory will be executed for scrubbing the data.
TODO
- Support specifying eppic macros in the makedumpfile.conf file. A new command should be added to makedumpfile.conf to identify an eppic macro. This command could be used in any section of the makedumpfile.conf, where the section name indicates the kernel module name. makedumpfile will only search for symbols encountered by eppic macro only in the specified module. Because, searching these symbols in all the modules will cause huge performance overhead.
--- erase_info.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++-- erase_info.h | 4 ++++ extension_eppic.h | 3 +++ makedumpfile.c | 7 ++++++- makedumpfile.h | 6 ++++++ 5 files changed, 67 insertions(+), 3 deletions(-)
erase_info.c +48 -2
@@ -1811,6 +1811,34 @@ process_config_file(const char *name_config)return TRUE; } + +/* Process the eppic macro using eppic library */ +#ifdef EPPIC +static int +process_eppic_file (char *name_config) +{ +int ret =0 ; + +/* Initialize eppic*/ + ret =eppic_init (); +if (ret) { +ERRMSG ("Eppic initialization failed\n" ); +return FALSE; + } + +/* TODO +* Support specifying eppic macros in makedumpfile.conf file +*/ + +/* Load/compile and execute the eppic macro */ +eppic_load (name_config); + +eppic_unload (name_config); + +return TRUE; +} +#endif +static void split_filter_info (struct filter_info *prev,unsigned long long next_paddr,size_t size)
@@ -1904,7 +1932,7 @@ extract_filter_info(unsigned long long start_paddr,int gather_filter_info (void ) { -int ret; +int ret = TRUE;/* * Before processing filter config file, load the symbol data of
@@ -1915,7 +1943,25 @@ gather_filter_info(void)if (!load_module_symbols ())return FALSE; - ret =process_config_file (info->name_filterconfig); +/* +* This patch supports specifying both makedumpfile.conf and +* eppic macro at the same time. Whether to retain or discard the +* functionality provided by makedumpfile.conf is open for +* discussion +*/ +if (info->name_filterconfig) + ret =process_config_file (info->name_filterconfig); + + if (info->name_eppic_config) +#ifdef EPPIC + ret &=process_eppic_file (info->name_eppic_config); +#else + { +MSG ("'-eppic' option is disabled, because this " ); +MSG ("binary doesn't support scrubbing using eppic.\n" ); +MSG ("Try `make EPPIC=on` when building.\n" ); + } +#endif /* * Remove modules symbol information, we dont need now.
erase_info.h +4 -0
@@ -19,6 +19,10 @@#ifndef _ERASE_INFO_H#define _ERASE_INFO_H +#ifdef EPPIC +#include "extension_eppic.h" +#endif +#define MAX_SIZE_STR_LEN (26)/*
extension_eppic.h +3 -0
@@ -20,4 +20,7 @@#include "eppic_api.h" +int eppic_init (void );/* Eppic initialize */ +#endif /* _EXTENSION_EPPIC_H */ +
makedumpfile.c +6 -1
@@ -6943,7 +6943,8 @@ retry: } } -if (info->name_filterconfig && !gather_filter_info ()) +if ((info->name_filterconfig || info->name_eppic_config) + && !gather_filter_info ())return FALSE;if (!create_dump_bitmap ())
@@ -7779,6 +7780,7 @@ static struct option longopts[] = { {"diskset", required_argument, NULL, 'k'}, {"non-cyclic", no_argument, NULL, 'Y'}, {"cyclic-buffer", required_argument, NULL, 'Z'}, + {"eppic", required_argument, NULL, 'S'}, {0, 0, 0, 0} };
@@ -7874,6 +7876,9 @@ main(int argc, char *argv[])case 's' : info->flag_split =1 ;break ; +case 'S' : + info->name_eppic_config = optarg; +break ;case 'r' : info->flag_reassemble =1 ;break ;
makedumpfile.h +6 -0
@@ -830,6 +830,12 @@ struct DumpInfo {FILE *file_filterconfig;/* +* Filter config file containing eppic language filtering rules +* to filter out kernel data from vmcore +*/ +char *name_eppic_config; + +/* * diskdimp info: */ int block_order;
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 3/7 Eppic call back functions to query a dump image
In reply to Aravinda PrasadThis patch implements a series of apigetuint* call back functions which are used to access data from the dump image. Eppic uses these call back functions to fetch the actual value of the global variables.
This patch also adds other call back functions as a place holder which will be implemented in later patches. This is mainly to avoid compilation error while registering the call back functions using eppic_apiset() function.
--- extension_eppic.c | 128 +++++++++++++++++++++++++++++++++++++++++++++++++++++ extension_eppic.h | 50 ++++++++++++++++++++- 2 files changed, 176 insertions(+), 2 deletions(-)
extension_eppic.c +127 -1
@@ -20,6 +20,7 @@#include <sys/types.h> #include <fcntl.h> +#include "makedumpfile.h" #include "extension_eppic.h" /*
@@ -64,6 +65,131 @@ reg_callback(char *name, int load)return ; } +/* +* Call back functions for eppic to query the dump image +*/ + +static int +apigetmem (ull iaddr,void *p,int nbytes) +{ +return readmem (VADDR , iaddr, p, nbytes); +} + +static int +apiputmem (ull iaddr,void *p,int nbytes) +{ +return 1 ; +} + +static char * +apimember (char *mname,ull pidx,type_t *tm, +member_t *m,ull *lidx) +{ +return 0 ; +} + +static int +apigetctype (int ctype,char *name,type_t *tout) +{ +return 0 ; +} + +static char * +apigetrtype (ull idx,type_t *t) +{ +return "" ; +} + +static int +apialignment (ull idx) +{ +return 0 ; +} + +int +apigetval (char *name,ull *val,VALUE_S *value) +{ +ull ptr =0 ; + + ptr =get_symbol_addr (name); +if (!ptr) +return 0 ; + + *val = ptr; +return 1 ; +} + +static enum_t * +apigetenum (char *name) +{ +return 0 ; +} + +static def_t * +apigetdefs (void ) +{ +return 0 ; +} + +static uint8_t +apigetuint8 (void *ptr) +{ +uint8_t val; +if (!readmem (VADDR , (unsigned long )ptr, (char *)&val,sizeof (val))) +return (uint8_t )-1 ; +return val; +} + +static uint16_t +apigetuint16 (void *ptr) +{ +uint16_t val; +if (!readmem (VADDR , (unsigned long )ptr, (char *)&val,sizeof (val))) +return (uint16_t )-1 ; +return val; +} + +static uint32_t +apigetuint32 (void *ptr) +{ +uint32_t val; +if (!readmem (VADDR , (unsigned long )ptr, (char *)&val,sizeof (val))) +return (uint32_t )-1 ; +return val; +} + +static uint64_t +apigetuint64 (void *ptr) +{ +uint64_t val; +if (!readmem (VADDR , (unsigned long )ptr, (char *)&val,sizeof (val))) +return (uint64_t )-1 ; +return val; +} + +static char * +apifindsym (char *p) +{ +return NULL ; +} + +apiops icops = { + apigetmem, + apiputmem, + apimember, + apigetctype, + apigetrtype, + apialignment, + apigetval, + apigetenum, + apigetdefs, + apigetuint8, + apigetuint16, + apigetuint32, + apigetuint64, + apifindsym +}; +/* Initialize eppic */ int
@@ -72,7 +198,7 @@ eppic_init()if (eppic_open () >=0 ) {/* Register call back functions */ -eppic_apiset (NULL ,3 ,sizeof (long ),0 ); +eppic_apiset (&icops,3 ,sizeof (long ),0 );/* set the new function callback */ eppic_setcallback (reg_callback);
extension_eppic.h +49 -1
54 lines not on this page · raw message
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 4/7 Implement apigetctype call back function
In reply to Aravinda Prasadlibeppic will call apigetctype call back function whenever it encounters a token in the eppic macro. The call back function will use DWARF to query information related to the requested token and will pass it back to eppic using libeppic API calls. If the token does not exist, then apigetctype call returns 0.
--- dwarf_info.c | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++ dwarf_info.h | 9 ++++ extension_eppic.c | 28 +++++++++++++- 3 files changed, 145 insertions(+), 1 deletions(-)
dwarf_info.c +109 -0
@@ -51,6 +51,7 @@ struct dwarf_info {long enum_number;/* OUT */ unsigned char type_flag;/* OUT */ char src_name[LEN_SRCFILE ];/* OUT */ +Dwarf_Off die_offset;/* OUT */ };static struct dwarf_info dwarf_info;
@@ -102,6 +103,22 @@ is_search_typedef(int cmd) }static int +is_search_domain (int cmd) +{ +if ((cmd ==DWARF_INFO_GET_DOMAIN_STRUCT ) + || (cmd ==DWARF_INFO_GET_DOMAIN_TYPEDEF ) + || (cmd ==DWARF_INFO_GET_DOMAIN_ARRAY ) + || (cmd ==DWARF_INFO_GET_DOMAIN_UNION ) + || (cmd ==DWARF_INFO_GET_DOMAIN_ENUM ) + || (cmd ==DWARF_INFO_GET_DOMAIN_REF ) + || (cmd ==DWARF_INFO_GET_DOMAIN_STRING ) + || (cmd ==DWARF_INFO_GET_DOMAIN_BASE )) +return TRUE; +else +return FALSE; +} + +static int process_module (Dwfl_Module *dwflmod, void **userdata __attribute__ ((unused)), const char *name __attribute__ ((unused)),
@@ -736,6 +753,74 @@ search_symbol(Dwarf_Die *die, int *found) }static void +search_domain (Dwarf_Die *die,int *found) +{ +int tag; +const char *name; +short flag =0 ; +Dwarf_Die die_type; + +do { + tag =dwarf_tag (die); + name =dwarf_diename (die); + +/* +* Descend into anonymous members and search for the +* needed domain there. +*/ +if (!name) { +if (!get_die_type (die, &die_type)) +continue ; + +if (is_anonymous_container (&die_type)) { +Dwarf_Die child; + +if (dwarf_child (&die_type, &child) !=0 ) +continue ; + +search_domain (&child, found); + +if (*found) +return ; + } + } + +if ((!name) ||strcmp (name, dwarf_info.symbol_name)) +continue ; + +switch (dwarf_info.cmd) { +case DWARF_INFO_GET_DOMAIN_STRUCT : +if (tag == DW_TAG_structure_type) + flag =1 ; +break ; +case DWARF_INFO_GET_DOMAIN_UNION : +if (tag == DW_TAG_union_type) + flag =1 ; +break ; +case DWARF_INFO_GET_DOMAIN_TYPEDEF : +if (tag == DW_TAG_typedef) + flag =1 ; +break ; +/* TODO +* Implement functionality for the rest of the domains +*/ + } + +if (!flag) +continue ; + + dwarf_info.struct_size =dwarf_bytesize (die); + +if (dwarf_info.struct_size >0 ) { +if (found) + *found = TRUE; + dwarf_info.die_offset =dwarf_dieoffset (die); +break ; + } + }while (!dwarf_siblingof (die, die)); +} + +static void search_die_tree (Dwarf_Die *die,int *found) {Dwarf_Die child;
@@ -760,6 +845,9 @@ search_die_tree(Dwarf_Die *die, int *found)else if (is_search_typedef (dwarf_info.cmd))search_typedef (die, found); + +else if (is_search_domain (dwarf_info.cmd)) +search_domain (die, found); } static int
@@ -1145,6 +1233,27 @@ get_source_filename(char *structname, char *src_name, int cmd) }/* +* Get the domain information of the symbol +*/ +long +get_domain (char *symname,int cmd,unsigned long long *die) +{ + dwarf_info.cmd = cmd; + dwarf_info.symbol_name = symname; + dwarf_info.type_name =NULL ; + dwarf_info.struct_size =NOT_FOUND_STRUCTURE ; + dwarf_info.die_offset =0 ; + +if (!get_debug_info ()) +return 0 ; + +if (die) + *die = (unsigned long long ) dwarf_info.die_offset; + +return dwarf_info.struct_size; +} + +/* * Set the dwarf_info with kernel/module debuginfo file information. */ int
dwarf_info.h +9 -0
@@ -46,6 +46,14 @@ enum {DWARF_INFO_CHECK_SYMBOL_ARRAY_TYPE ,DWARF_INFO_GET_SYMBOL_TYPE ,DWARF_INFO_GET_MEMBER_TYPE , +DWARF_INFO_GET_DOMAIN_STRUCT , +DWARF_INFO_GET_DOMAIN_TYPEDEF , +DWARF_INFO_GET_DOMAIN_ARRAY , +DWARF_INFO_GET_DOMAIN_UNION , +DWARF_INFO_GET_DOMAIN_ENUM , +DWARF_INFO_GET_DOMAIN_REF , +DWARF_INFO_GET_DOMAIN_STRING , + DWARF_INFO_GET_DOMAIN_BASE, };char *get_dwarf_module_name (void );
@@ -60,6 +68,7 @@ char *get_member_type_name(char *structname, char *membername, int cmd, long *silong get_array_length (char *name01,char *name02,unsigned int cmd);long get_enum_number (char *enum_name);int get_source_filename (char *structname,char *src_name,int cmd); +long get_domain (char *symname,int cmd,unsigned long long *die);int set_dwarf_debuginfo (char *mod_name,char *os_release,char *name_debuginfo,int fd_debuginfo);#endif /* DWARF_INFO_H */
extension_eppic.c +27 -1
@@ -91,7 +91,33 @@ apimember(char *mname, ull pidx, type_t *tm,static int apigetctype (int ctype,char *name,type_t *tout) { -return 0 ; +long size =0 ; +unsigned long long die =0 ; + +switch (ctype) { +case V_TYPEDEF : + size =get_domain (name,DWARF_INFO_GET_DOMAIN_TYPEDEF , &die); +break ; +case V_STRUCT : + size =get_domain (name,DWARF_INFO_GET_DOMAIN_STRUCT , &die); +break ; +case V_UNION : + size =get_domain (name,DWARF_INFO_GET_DOMAIN_UNION , &die); +break ; +/* TODO +* Implement for all the domains +*/ + } + +if (size <=0 || !die) +return 0 ; + +/* populate */ +eppic_type_settype (tout, ctype); +eppic_type_setsize (tout, size); +eppic_type_setidx (tout, (ull )(unsigned long )die); +eppic_pushref (tout,0 ); +return 1 ; } static char *
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 5/7 Implement apimember and apigetrtype call back functions
In reply to Aravinda PrasadThe patch includes functionality for apimember and apigetrtype eppic callback routines along with helper functions to fetch information related to the member of the structure/union
Whenever a structure/union member is accessed inside the eppic macro, eppic will query makedumpfile through call back functions requesting more information on the structure or union member. The information includes member name, offset from structure, data type and size. makedumpfile will get all these information using DWARF and pass back to eppic using libeppic API calls.
--- dwarf_info.c | 206 +++++++++++++++++++++++++++++++++++++++++++++++++++++ dwarf_info.h | 7 ++ extension_eppic.c | 179 +++++++++++++++++++++++++++++++++++++++++++++- extension_eppic.h | 2 + 4 files changed, 390 insertions(+), 4 deletions(-)
dwarf_info.c +206 -0
@@ -463,6 +463,29 @@ get_dwarf_base_type(Dwarf_Die *die)return TRUE; } +/* +* Get the die, given the offset +*/ +static int +get_die_from_offset (Dwarf_Off offset,Dwarf_Die *die) +{ +if (!init_dwarf_info ()) +return FALSE; + +if ((!offset) || (!die)) +return FALSE; + +if (!dwarf_offdie (dwarf_info.dwarfd, offset, die)) { +ERRMSG ("Can't find the DIE.\n" ); +return FALSE; + } + +return TRUE; +} + +/* +* Function for searching struct page.union.struct.mapping. +*/ static int is_anonymous_container (Dwarf_Die *die) {
@@ -1254,6 +1277,189 @@ get_domain(char *symname, int cmd, unsigned long long *die) }/* +* Get the number of fields in a structure or union provided the +* die offset of the structure or union +*/ +int +get_die_nfields (unsigned long long die_off) +{ +int tag, nfields =0 ; +Dwarf_Die result, child, *die; + +if (!get_die_from_offset ((Dwarf_Off ) die_off, &result)) { +ERRMSG ("Can't find the DIE.\n" ); +return -1 ; + } + + die = &result; + tag =dwarf_tag (die); +if (tag != DW_TAG_structure_type && tag != DW_TAG_union_type) { +ERRMSG ("DIE is not of structure or union type.\n" ); +return -1 ; + } + +if (dwarf_child (die, &child) !=0 ) +return -1 ; + +/* Find the number of fields in the structure */ + die = &child; +do { + tag =dwarf_tag (die); +if (tag == DW_TAG_member) + nfields++; +else +continue ; + }while (!dwarf_siblingof (die, die)); + +return nfields; +} + +/* +* Get the information of the structure member given by index +*/ +int +get_die_member (unsigned long long die_off,int index,long *offset, +char **name,int *nbits,int *fbits,unsigned long long *m_die) +{ +int tag, size, nfields =0 ; +Dwarf_Die result, child, die_base, *die; + +if (!offset || !nbits || !fbits || !name || !m_die) +return -1 ; + +if (!get_die_from_offset ((Dwarf_Off ) die_off, &result)) { +ERRMSG ("Can't find the DIE.\n" ); +return -1 ; + } + + die = &result; + tag =dwarf_tag (die); +if (tag != DW_TAG_structure_type && tag != DW_TAG_union_type) { +ERRMSG ("DIE is not of structure or union type.\n" ); +return -1 ; + } + +if (dwarf_child (die, &child) !=0 ) +return -1 ; + +/* Find the correct field in the structure */ + die = &child; +do { + tag =dwarf_tag (die); +if (tag == DW_TAG_member) { +if (nfields == index) +break ; +else + nfields++; + } + }while (!dwarf_siblingof (die, die)); + +if (nfields != index) { +ERRMSG ("No member found at index %d.\n" , index); +return -1 ; + } + +/* Fill in the required info for the member */ +if (!get_data_member_location (die, offset)) + *offset =0 ; + + *name =dwarf_diename (die); + *m_die =dwarf_dieoffset (die); + +get_die_type (die, &die_base); +if (dwarf_tag (&die_base) == DW_TAG_array_type) { + dwarf_info.array_length =0 ; +get_data_array_length (die); + size = dwarf_info.array_length; + }else { + size =dwarf_bytesize (&die_base); + } + +/* TODO +* Correctly update fbits and nbits +*/ + *nbits = *fbits =0 ; + +if (size <0 ) +return 0 ; +else +return size; +} + +/* +* Get the die attribute type +*/ +int +get_die_attr_type (unsigned long long die_off,int *type_flag, +unsigned long long *die_attr_off) +{ +Dwarf_Die result; + +if (!die_attr_off) +return FALSE; + +if (!get_die_from_offset ((Dwarf_Off ) die_off, &result)) { +ERRMSG ("Can't find the DIE.\n" ); +return FALSE; + } + +if (!get_die_type (&result, &result)) +return FALSE; + + *die_attr_off =dwarf_dieoffset (&result); + *type_flag =dwarf_tag (&result); +return TRUE; +} + +/* +* Get name attribute given the die offset +*/ +char * +get_die_name (unsigned long long die_off) +{ +Dwarf_Die result; + +if (!die_off) +return NULL ; + +if (!get_die_from_offset ((Dwarf_Off ) die_off, &result)) { +ERRMSG ("Can't find the DIE.\n" ); +return NULL ; + } + +return dwarf_diename (&result); +} + +/* +* Get length attribute given the die offset +*/ +int +get_die_length (unsigned long long die_off,int flag) +{ +Dwarf_Die result, die_base; + +if (!die_off) +return FALSE; + +if (!get_die_from_offset ((Dwarf_Off ) die_off, &result)) { +ERRMSG ("Can't find the DIE.\n" ); +return FALSE; + } + +if (flag) +return dwarf_bytesize (&result); + +get_die_type (&result, &die_base); +if (dwarf_tag (&die_base) == DW_TAG_array_type) { + dwarf_info.array_length =0 ; +get_data_array_length (&result); +return dwarf_info.array_length; + }else { +return dwarf_bytesize (&die_base); + } +} + +/* * Set the dwarf_info with kernel/module debuginfo file information. */ int
dwarf_info.h +7 -0
@@ -69,6 +69,13 @@ long get_array_length(char *name01, char *name02, unsigned int cmd);long get_enum_number (char *enum_name);int get_source_filename (char *structname,char *src_name,int cmd);long get_domain (char *symname,int cmd,unsigned long long *die); +int get_die_nfields (unsigned long long die_off); +int get_die_member (unsigned long long die_off,int index,long *offset, +char **name,int *nbits,int *fbits,unsigned long long *m_die); +int get_die_attr_type (unsigned long long die_off,int *type_flag, +unsigned long long *die_attr_off); +char *get_die_name (unsigned long long die_off); +int get_die_length (unsigned long long die_off,int flag);int set_dwarf_debuginfo (char *mod_name,char *os_release,char *name_debuginfo,int fd_debuginfo);#endif /* DWARF_INFO_H */
extension_eppic.c +175 -4
208 lines not on this page · raw message
extension_eppic.h +2 -0
@@ -29,6 +29,8 @@ int eppic_init(void); /* Eppic initialize */* for makedumpfile extension. */ +#define MAX_ARRAY_DIMENSION 16 +/* member information */ typedef MEMBER_S {
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 6/7 Extend eppic built-in functions to include memset function
In reply to Aravinda PrasadThe memset function will be used to specify the virtual address and the length of the data to be scrubbed in the dump file from the eppic macro. makedumpfile will convert these requests into filter_info nodes which will be enqueued for filtering. Existing makedumpfile functionality reads the filter_info nodes and scrubs the data accordingly.
--- erase_info.c | 33 ++++++++++++++++++++++++++++++++- erase_info.h | 1 + extension_eppic.c | 20 ++++++++++++++++++++ extension_eppic.h | 2 ++ 4 files changed, 55 insertions(+), 1 deletions(-)
erase_info.c +32 -1
@@ -66,6 +66,8 @@ struct filter_info {int erase_info_idx;/* 0= invalid index */ int size_idx; +int erase_ch; +struct filter_info *next;unsigned short nullify; };
@@ -1574,6 +1576,7 @@ update_filter_info(struct config_entry *filter_symbol, fl_info->paddr =vaddr_to_paddr (sym_addr); fl_info->size = size; fl_info->nullify = filter_symbol->nullify; + fl_info->erase_ch ='X' ;if (insert_filter_info (fl_info)) { fl_info->erase_info_idx =add_erase_info_node (filter_symbol);
@@ -1582,6 +1585,34 @@ update_filter_info(struct config_entry *filter_symbol,return TRUE; } +int +update_filter_info_raw (unsigned long long sym_addr,int ch,int len) +{ +struct filter_info *fl_info; + + fl_info =calloc (1 ,sizeof (struct filter_info )); +if (fl_info ==NULL ) { +ERRMSG ("Can't allocate filter info\n" ); +return FALSE; + } + + fl_info->vaddr = sym_addr; + fl_info->paddr =vaddr_to_paddr (sym_addr); + fl_info->size = len; + fl_info->nullify =0 ; + fl_info->erase_ch = ch; + +if (insert_filter_info (fl_info)) { +/* TODO +* Add support to update erase information to the +* resulting dump file +*/ + fl_info->erase_info_idx =0 ; + fl_info->size_idx =0 ; + } +return TRUE; +} +static int initialize_iteration_entry (struct config_entry *ie,char *type_name,unsigned char type_flag)
@@ -2014,7 +2045,7 @@ filter_data_buffer(unsigned char *buf, unsigned long long paddr,if (fl_info.nullify)memset (buf_ptr,0 , fl_info.size);else -memset (buf_ptr,'X' , fl_info.size); +memset (buf_ptr, fl_info.erase_ch, fl_info.size); } }
erase_info.h +1 -0
@@ -42,6 +42,7 @@ int gather_filter_info(void);void clear_filter_info (void );void filter_data_buffer (unsigned char *buf,unsigned long long paddr,size_t size);unsigned long get_size_eraseinfo (void ); +int update_filter_info_raw (unsigned long long ,int ,int );#endif /* _ERASE_INFO_H */
extension_eppic.c +20 -0
@@ -387,6 +387,22 @@ apiops icops = { apifindsym }; +/* Extensions to built-in functions */ +VALUE_S * +eppic_memset (VALUE_S *vaddr,VALUE_S *vch,VALUE_S *vlen) +{ +ull addr =eppic_getval (vaddr); +int len =eppic_getval (vlen); +int ch =eppic_getval (vch); + +/* +* Set the value at address from iaddr till iaddr + nbytes +* to the value specified in variable ch +*/ +update_filter_info_raw (addr, ch, len); +return eppic_makebtype (1 ); +} +/* Initialize eppic */ int
@@ -400,6 +416,10 @@ eppic_init()/* set the new function callback */ eppic_setcallback (reg_callback); +/* Extend built-in functions to include memset */ +eppic_builtin ("int memset(char *, int, int)" , + (bf_t *)eppic_memset); +return 0 ; }return 1 ;
extension_eppic.h +2 -0
@@ -73,4 +73,6 @@ typedef TYPE_S { ull rtype; /* type_t a reference refers too */ } type_t; +extern int update_filter_info_raw (unsigned long long ,int ,int ); +#endif /* _EXTENSION_EPPIC_H */
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 7/7 Support fully typed symbol access mode
In reply to Aravinda PrasadEppic enables access to symbols in two ways. The first, is a more natural mode in that it makes symbols available as fully typed entities. The second, is generic and treats all symbols as an address to data which then needs to be cast to the proper type explicitly. The former obviously enables an easier cut & pasting of target code into eppic code.
Currently generic symbol access mode is supported. This patch extends the functionality to include support for fully typed symbol access mode (which will be the default mode) in eppic macros. User can switch to generic symbol access mode by setting the following environmental variable - EPPIC_LEGACY_MODE. libeppic.a will take care of handling EPPIC_LEGACY_MODE. libeppic.a will pass NULL to VALUE_S* argument of apigetval() call back function if EPPIC_LEGACY_MODE is set.
Refer to http://code.google.com/p/eppic/ for more information.
--- dwarf_info.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ dwarf_info.h | 2 ++ extension_eppic.c | 24 ++++++++++++++++++++++++ 3 files changed, 78 insertions(+), 0 deletions(-)
dwarf_info.c +52 -0
@@ -119,6 +119,15 @@ is_search_domain(int cmd) }static int +is_search_die (int cmd) +{ +if (cmd ==DWARF_INFO_GET_DIE ) +return TRUE; +else +return FALSE; +} + +static int process_module (Dwfl_Module *dwflmod, void **userdata __attribute__ ((unused)), const char *name __attribute__ ((unused)),
@@ -844,6 +853,25 @@ search_domain(Dwarf_Die *die, int *found) }static void +search_die (Dwarf_Die *die,int *found) +{ +const char *name; + +do { + name =dwarf_diename (die); + +if ((!name) ||strcmp (name, dwarf_info.symbol_name)) +continue ; + +if (found) + *found = TRUE; + + dwarf_info.die_offset =dwarf_dieoffset (die); +return ; + }while (!dwarf_siblingof (die, die)); +} + +static void search_die_tree (Dwarf_Die *die,int *found) {Dwarf_Die child;
@@ -871,6 +899,9 @@ search_die_tree(Dwarf_Die *die, int *found)else if (is_search_domain (dwarf_info.cmd))search_domain (die, found); + +else if (is_search_die (dwarf_info.cmd)) +search_die (die, found); } static int
@@ -1431,6 +1462,27 @@ get_die_name(unsigned long long die_off) }/* +* Get the die offset given the die name +*/ +unsigned long long +get_die_offset (char *sysname) +{ + dwarf_info.cmd =DWARF_INFO_GET_DIE ; + dwarf_info.symbol_name = sysname; + dwarf_info.type_name =NULL ; + dwarf_info.struct_size =NOT_FOUND_STRUCTURE ; + dwarf_info.die_offset =0 ; + +if (!sysname) +return 0 ; + +if (!get_debug_info ()) +return 0 ; + +return (unsigned long long )dwarf_info.die_offset; +} + +/* * Get length attribute given the die offset */ int
dwarf_info.h +2 -0
@@ -54,6 +54,7 @@ enum {DWARF_INFO_GET_DOMAIN_REF ,DWARF_INFO_GET_DOMAIN_STRING ,DWARF_INFO_GET_DOMAIN_BASE , + DWARF_INFO_GET_DIE, };char *get_dwarf_module_name (void );
@@ -75,6 +76,7 @@ int get_die_member(unsigned long long die_off, int index, long *offset,int get_die_attr_type (unsigned long long die_off,int *type_flag,unsigned long long *die_attr_off);char *get_die_name (unsigned long long die_off); +unsigned long long get_die_offset (char *sysname);int get_die_length (unsigned long long die_off,int flag);int set_dwarf_debuginfo (char *mod_name,char *os_release,char *name_debuginfo,int fd_debuginfo);
extension_eppic.c +24 -0
@@ -313,6 +313,30 @@ apigetval(char *name, ull *val, VALUE_S *value)return 0 ; *val = ptr; + +if (!value) +return 1 ; + +/* Support for fully typed symbol access */ +ull type; +TYPE_S *stype; + + type =get_die_offset (name); + stype =eppic_gettype (value); + +apigetrtype (type, stype); + +eppic_pushref (stype,1 ); +eppic_setmemaddr (value, *val); +eppic_do_deref (1 , value, value); + + *val =eppic_getval (value); + +if (!eppic_typeislocal (stype) &&eppic_type_getidx (stype) >100 ) { +char *tname =get_die_name (eppic_type_getidx (stype)); +if (tname) +eppic_chktype (stype, tname); + }return 1 ; }
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello Aravinda,
Quoted 21 lines · On Thu, 08 Nov 2012 19:07:52 +0530 Aravinda Prasad…
makedumpfile security key filtering enhancement - Add Eppic language support (formerly known as SIAL) to specify rules to scrub data in a dumpfile. Eppic was previously part of crash source code repository.
The following series of patches enhance the makedumpfile to provide a more powerful way to specify rules and commands to traverse and erase complex data structures in a dump file by integrating Embeddable Pre-Processor and Interpreter for C (eppic).
Eppic is an interpreter that facilitates access to the symbol and type information stored in an executable image or a dump file. Eppic defines a language semantic which is similar to C. Eppic macros can be used to specify rules/commands to erase data in an image file. makedumpfile will interpret the rules/commands provided by eppic macros with the help of eppic library and will suitably erase the required data in a dump file. Eppic provides a lot of language constructs like conditional statements, logical and arithmetic operators, nested loops, functions, etc., to traverse nested lists and trees and conditionally erase data in the dump file, enabling users to literally erase any data in the dump file which is accessible through global symbols.
Quoted 6 lines
The series of patches integrates eppic with makdumpfile. These patches require eppic library libeppic.a and
eppic_api.hheader file. The libeppic.a library can be built from the eppic source code available at the following URL:
Thank you for re-sending your patches, I'll merge them into v1.5.1-rc. And thanks for sending your test script, it's helpful for understanding how to use eppic macro.
Atsushi Kumagai
Quoted 35 lines
TODO:
- Currently, works only for symbols in vmlinux, extend it to module symbols
- Functionality support:
- Implement the following callback functions.
- apialignment
- apigetenum
- apigetdefs
- Other functionalities specified in the code with TODO tag
- Support specifying eppic macros in makedumpfile.conf file
- Update erase infoChangelog from v1 to v2:
- Re-based to v1.5.0
- Introduced EPPIC=on in makefile, and hence eppic is now optional
- Incorporated review comments from Atsushi
- Minor formatting changesRegards,
Aravinda
---Aravinda Prasad (7):
Initialize and setup eppic
makedumpfile and eppic interface layer
Eppic call back functions to query a dump image Implement apigetctype call back function Implement apimember and apigetrtype call back functions Extend eppic built-in functions to include memset function Support fully typed symbol access mode
Quoted 21 lines
Makefile | 7 + dwarf_info.c | 367 +++++++++++++++++++++++++++++++++++++++++++ dwarf_info.h | 18 ++ erase_info.c | 83 +++++++++- erase_info.h | 5 + extension_eppic.c | 451 +++++++++++++++++++++++++++++++++++++++++++++++++++++ extension_eppic.h | 78 +++++++++ makedumpfile.c | 7 + makedumpfile.h | 6 + 9 files changed, 1018 insertions(+), 4 deletions(-) create mode 100644 extension_eppic.c create mode 100644 extension_eppic.hSignature · 7 lines
Aravinda Prasad _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 8 lines · Aravinda Prasad
makedumpfile security key filtering enhancement - Add Eppic language support (formerly known as SIAL) to specify rules to scrub data in a dumpfile. Eppic was previously part of crash source code repository.
The following series of patches enhance the makedumpfile to provide a more powerful way to specify rules and commands to traverse and erase complex data structures in a dump file by integrating Embeddable Pre-Processor and Interpreter for C (eppic).
Hi Aravinda,
Had few questions.
- Which file will contain all the rules?
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
- Is this supposed to work from kdump initramfs or it is supposed to be used on already saved dump (later during post processing).
Given the fact that it does not reduce the size of core file significantly, I would think that it is better to post process vmcore to wipe out some symbols.
Thanks
Vivek
Quoted 54 lines
Eppic is an interpreter that facilitates access to the symbol and type information stored in an executable image or a dump file. Eppic defines a language semantic which is similar to C. Eppic macros can be used to specify rules/commands to erase data in an image file. makedumpfile will interpret the rules/commands provided by eppic macros with the help of eppic library and will suitably erase the required data in a dump file. Eppic provides a lot of language constructs like conditional statements, logical and arithmetic operators, nested loops, functions, etc., to traverse nested lists and trees and conditionally erase data in the dump file, enabling users to literally erase any data in the dump file which is accessible through global symbols.
The series of patches integrates eppic with makdumpfile. These patches require eppic library libeppic.a and
eppic_api.hheader file. The libeppic.a library can be built from the eppic source code available at the following URL:http://code.google.com/p/eppic/
TODO:
- Currently, works only for symbols in vmlinux, extend it to module symbols
- Functionality support:
- Implement the following callback functions.
- apialignment
- apigetenum
- apigetdefs
- Other functionalities specified in the code with TODO tag
- Support specifying eppic macros in makedumpfile.conf file
- Update erase infoChangelog from v1 to v2:
- Re-based to v1.5.0
- Introduced EPPIC=on in makefile, and hence eppic is now optional
- Incorporated review comments from Atsushi
- Minor formatting changesRegards,
Aravinda
---Aravinda Prasad (7):
Initialize and setup eppic
makedumpfile and eppic interface layer
Eppic call back functions to query a dump image Implement apigetctype call back function Implement apimember and apigetrtype call back functions Extend eppic built-in functions to include memset function Support fully typed symbol access mode
Quoted 21 lines
Makefile | 7 + dwarf_info.c | 367 +++++++++++++++++++++++++++++++++++++++++++ dwarf_info.h | 18 ++ erase_info.c | 83 +++++++++- erase_info.h | 5 + extension_eppic.c | 451 +++++++++++++++++++++++++++++++++++++++++++++++++++++ extension_eppic.h | 78 +++++++++ makedumpfile.c | 7 + makedumpfile.h | 6 + 9 files changed, 1018 insertions(+), 4 deletions(-) create mode 100644 extension_eppic.c create mode 100644 extension_eppic.hSignature · 7 lines
Aravinda Prasad _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hi Vivek,
Quoted 10 lines · Vivek Goyal
Quoted 8 lines · Aravinda Prasad
makedumpfile security key filtering enhancement - Add Eppic language support (formerly known as SIAL) to specify rules to scrub data in a dumpfile. Eppic was previously part of crash source code repository.
The following series of patches enhance the makedumpfile to provide a more powerful way to specify rules and commands to traverse and erase complex data structures in a dump file by integrating Embeddable Pre-Processor and Interpreter for C (eppic).
Quoted 5 lines
Hi Aravinda,
Had few questions.
- Which file will contain all the rules?
As of now rule files will not be provided by makedumpfile. However, writing a rule file is very easy - it is a C program.
Quoted 4 lines
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
Memory footprint of libeppic.a is around 1MB. Yes, this will be statically linked to makedumpfile. Users should specify EPPIC=on while building the makedumpfile and hence linking libeppic.a is optional
Quoted 3 lines
- Is this supposed to work from kdump initramfs or it is supposed to be used on already saved dump (later during post processing).
For the time being, it is only during post processing.
Quoted 4 lines
Given the fact that it does not reduce the size of core file significantly, I would think that it is better to post process vmcore to wipe out some symbols.
The main intention is to remove confidential information from the dump file, like ssh keys etc., which could be just few bytes, hence, may not reduce the size of the dump significantly.
Quoted 80 lines
Thanks
VivekQuoted 75 lines
Eppic is an interpreter that facilitates access to the symbol and type information stored in an executable image or a dump file. Eppic defines a language semantic which is similar to C. Eppic macros can be used to specify rules/commands to erase data in an image file. makedumpfile will interpret the rules/commands provided by eppic macros with the help of eppic library and will suitably erase the required data in a dump file. Eppic provides a lot of language constructs like conditional statements, logical and arithmetic operators, nested loops, functions, etc., to traverse nested lists and trees and conditionally erase data in the dump file, enabling users to literally erase any data in the dump file which is accessible through global symbols.
The series of patches integrates eppic with makdumpfile. These patches require eppic library libeppic.a and
eppic_api.hheader file. The libeppic.a library can be built from the eppic source code available at the following URL:http://code.google.com/p/eppic/
TODO:
- Currently, works only for symbols in vmlinux, extend it to module symbols
- Functionality support:
- Implement the following callback functions.
- apialignment
- apigetenum
- apigetdefs
- Other functionalities specified in the code with TODO tag
- Support specifying eppic macros in makedumpfile.conf file
- Update erase infoChangelog from v1 to v2:
- Re-based to v1.5.0
- Introduced EPPIC=on in makefile, and hence eppic is now optional
- Incorporated review comments from Atsushi
- Minor formatting changesRegards,
Aravinda
---Aravinda Prasad (7):
Initialize and setup eppic
makedumpfile and eppic interface layer
Eppic call back functions to query a dump image Implement apigetctype call back function Implement apimember and apigetrtype call back functions Extend eppic built-in functions to include memset function Support fully typed symbol access modeMakefile | 7 + dwarf_info.c | 367 +++++++++++++++++++++++++++++++++++++++++++ dwarf_info.h | 18 ++ erase_info.c | 83 +++++++++- erase_info.h | 5 + extension_eppic.c | 451 +++++++++++++++++++++++++++++++++++++++++++++++++++++ extension_eppic.h | 78 +++++++++ makedumpfile.c | 7 + makedumpfile.h | 6 + 9 files changed, 1018 insertions(+), 4 deletions(-) create mode 100644 extension_eppic.c create mode 100644 extension_eppic.hSignature · 7 lines
Aravinda Prasad _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 16 lines · Aravinda Prasad
Hi Vivek,
Quoted 12 lines · Vivek Goyal
Quoted 8 lines · Aravinda Prasad
makedumpfile security key filtering enhancement - Add Eppic language support (formerly known as SIAL) to specify rules to scrub data in a dumpfile. Eppic was previously part of crash source code repository.
The following series of patches enhance the makedumpfile to provide a more powerful way to specify rules and commands to traverse and erase complex data structures in a dump file by integrating Embeddable Pre-Processor and Interpreter for C (eppic).
Hi Aravinda,
Quoted 7 lines
Quoted 3 lines
Had few questions.
- Which file will contain all the rules?
As of now rule files will not be provided by makedumpfile. However, writing a rule file is very easy - it is a C program.
Can you give some details about how does it work and what's the correlation with makedumpfile.
Quoted 10 lines
Quoted 5 lines
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
Memory footprint of libeppic.a is around 1MB. Yes, this will be statically linked to makedumpfile. Users should specify EPPIC=on while building the makedumpfile and hence linking libeppic.a is optional
How would distributions handle it. Will we continue to build makedumpfile without EPPIC=on. Any increase in initramfs size increase is frowned upon in general.
Quoted 7 lines
Quoted 4 lines
- Is this supposed to work from kdump initramfs or it is supposed to be used on already saved dump (later during post processing).
For the time being, it is only during post processing.
Again, how distributions will handle it. If it is being integrated makedumpfile, as opposed to an stand alone utility, that means it makedumpfile needs to link against this library so that somebody can later filter out the symbols. And that means initramfs size bloat too?
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 19 lines · Vivek Goyal
Quoted 18 lines · Aravinda Prasad
Hi Vivek,
Quoted 14 lines · Vivek Goyal
Quoted 8 lines · Aravinda Prasad
makedumpfile security key filtering enhancement - Add Eppic language support (formerly known as SIAL) to specify rules to scrub data in a dumpfile. Eppic was previously part of crash source code repository.
The following series of patches enhance the makedumpfile to provide a more powerful way to specify rules and commands to traverse and erase complex data structures in a dump file by integrating Embeddable Pre-Processor and Interpreter for C (eppic).
Hi Aravinda,
Had few questions.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
- Which file will contain all the rules?
As of now rule files will not be provided by makedumpfile. However, writing a rule file is very easy - it is a C program.
Can you give some details about how does it work and what's the correlation with makedumpfile.
struct key in include/linux/key.h holds "authentication token"/"access credential"/"keyring". Suppose these entries should be scrubbed from the dumpfile. Then the keyring_name_hash hash table should be scanned and for each non-empty list, the entire list should be traversed and payload.value (or any other data) in struct key should be cleared.
Now the EPPIC macro looks like this:
intkey()
{
int i;
struct list_head *head;
struct list_head *next, *prev;
head = (struct list_head *)keyring_name_hash;
for (i = 0; i < 32; i++)
{
next = (struct list_head *) head[i].next;
prev = (struct list_head *) head[i].prev;
if (!next)
continue;
do
{
struct key *mykey, *off = 0;
mykey = (struct key *)((unsigned long)(next)
- ((unsigned long)&(off->type_data)));
memset((char *)mykey->payload.value, 'X', 0x8);
next = *(struct list_head **) mykey->type_data.link.next;
} while (next != prev);
}
return 1;
}The data can be cleared by specifying:
makedumpfile -c -d 31 -x vmlinux --eppic key.c vmcore filtered_vmcore
makedumpfile with the help of eppic will interpret the macro key.c, traverses all the hash chains and erases paylod.value of struct key.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
Quoted 7 lines
Quoted 3 lines
Memory footprint of libeppic.a is around 1MB. Yes, this will be statically linked to makedumpfile. Users should specify EPPIC=on while building the makedumpfile and hence linking libeppic.a is optional
How would distributions handle it. Will we continue to build makedumpfile without EPPIC=on. Any increase in initramfs size increase is frowned upon in general.
We would like distributions to build makedumpfile with EPPIC=on.
I am not sure, but do you think ~1MB is too much increase to go with?
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
- Is this supposed to work from kdump initramfs or it is supposed to be used on already saved dump (later during post processing).
Quoted 6 lines
Quoted 1 line
For the time being, it is only during post processing.
Again, how distributions will handle it. If it is being integrated makedumpfile, as opposed to an stand alone utility, that means it makedumpfile needs to link against this library so that somebody can later filter out the symbols. And that means initramfs size bloat too?
Yes, makedumpfile needs to be linked against eppic library for filtering data and this will increase makedumpfile size and initramfs size too.
Quoted 9 lines
Thanks
Vivek_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
----- Original Message -----
Quoted 70 lines
struct key in
include/linux/key.hholds "authentication token"/"access credential"/"keyring". Suppose these entries should be scrubbed from the dumpfile. Then thekeyring_name_hashhash table should be scanned and for each non-empty list, the entire list should be traversed and payload.value (or any other data) in struct key should be cleared.Now the EPPIC macro looks like this:
int
key(){ int i; struct list_head *head; struct list_head *next, *prev; head = (struct list_head *)keyring_name_hash; for (i = 0; i < 32; i++) { next = (struct list_head *) head[i].next; prev = (struct list_head *) head[i].prev; if (!next) continue; do { struct key *mykey, *off = 0; mykey = (struct key *)((unsigned long)(next) - ((unsigned long)&(off->type_data))); memset((char *)mykey->payload.value, 'X', 0x8); next = *(struct list_head **) mykey->type_data.link.next; } while (next != prev); } return 1; }The data can be cleared by specifying:
makedumpfile -c -d 31 -x vmlinux --eppickey.cvmcorefiltered_vmcoremakedumpfile with the help of eppic will interpret the macro
key.c, traverses all the hash chains and erases paylod.value of struct key.Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
Quoted 7 lines
Quoted 3 lines
Memory footprint of libeppic.a is around 1MB. Yes, this will be statically linked to makedumpfile. Users should specify EPPIC=on while building the makedumpfile and hence linking libeppic.a is optional
How would distributions handle it. Will we continue to build makedumpfile without EPPIC=on. Any increase in initramfs size increase is frowned upon in general.
We would like distributions to build makedumpfile with EPPIC=on.
I am not sure, but do you think ~1MB is too much increase to go with?
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
- Is this supposed to work from kdump initramfs or it is supposed to be used on already saved dump (later during post processing).
For the time being, it is only during post processing.
Quoted 8 lines
Quoted 5 lines
Again, how distributions will handle it. If it is being integrated makedumpfile, as opposed to an stand alone utility, that means it makedumpfile needs to link against this library so that somebody can later filter out the symbols. And that means initramfs size bloat too?
Yes, makedumpfile needs to be linked against eppic library for filtering data and this will increase makedumpfile size and initramfs size too.
Just to clarify -- your example indicates that the vmlinux file is required for this facility to work, correct?
Quoted 1 line
makedumpfile -c -d 31 -x vmlinux --eppic
key.cvmcorefiltered_vmcore
Clearly distros won't be putting the vmlinux file in the initramfs -- that's the whole reasoning behind vmcoreinfo. So the 99% of users that aren't interested in scrubbing will have to pay the penalty of the larger makedumpfile binary.
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
On Thu, Nov 15, 2012 at 09:27:45AM -0500, Dave Anderson wrote:
[..]
Quoted 6 lines
Quoted 2 lines
Yes, makedumpfile needs to be linked against eppic library for filtering data and this will increase makedumpfile size and initramfs size too.
Just to clarify -- your example indicates that the vmlinux file is required for this facility to work, correct?
Quoted 6 lines
Quoted 1 line
makedumpfile -c -d 31 -x vmlinux --eppic
key.cvmcorefiltered_vmcoreClearly distros won't be putting the vmlinux file in the initramfs -- that's the whole reasoning behind vmcoreinfo. So the 99% of users that aren't interested in scrubbing will have to pay the penalty of the larger makedumpfile binary.
That's a good point Dave. We will never put debug compiled vmlinux in initramfs. Following two alternatives come to my mind.
- Either makedumpfile provides some kind of library to parse/read/write dump files and we can write another stand alone utility for scrubbing dump files (say, scrub-vmcore), and it can link against makedumpfile libraries to take advantage of existing code.
- Or, we just identify what we want to scrube and make that code part of makedumpfile. Export relevant data structures from kernel using vmcoreinfo.
I prefer to keep things simple and like second option better.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 11 lines · Vivek Goyal
On Thu, Nov 15, 2012 at 09:27:45AM -0500, Dave Anderson wrote:
[..]
Quoted 8 lines
Quoted 2 lines
Yes, makedumpfile needs to be linked against eppic library for filtering data and this will increase makedumpfile size and initramfs size too.
Just to clarify -- your example indicates that the vmlinux file is required for this facility to work, correct?
Quoted 1 line
makedumpfile -c -d 31 -x vmlinux --eppic
key.cvmcorefiltered_vmcore
Quoted 7 lines
Quoted 4 lines
Clearly distros won't be putting the vmlinux file in the initramfs -- that's the whole reasoning behind vmcoreinfo. So the 99% of users that aren't interested in scrubbing will have to pay the penalty of the larger makedumpfile binary.
That's a good point Dave. We will never put debug compiled vmlinux in initramfs. Following two alternatives come to my mind.
As I mentioned, we don't need vmlinux in initramfs as filtering is done during post processing only.
Quoted 15 lines
- Either makedumpfile provides some kind of library to parse/read/write dump files and we can write another stand alone utility for scrubbing dump files (say, scrub-vmcore), and it can link against makedumpfile libraries to take advantage of existing code.
- Or, we just identify what we want to scrube and make that code part of makedumpfile. Export relevant data structures from kernel using vmcoreinfo.
I prefer to keep things simple and like second option better.
Thanks
Vivek
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 20 lines · Aravinda Prasad
Quoted 16 lines · Vivek Goyal
On Thu, Nov 15, 2012 at 09:27:45AM -0500, Dave Anderson wrote:
[..]
Quoted 12 lines
Quoted 2 lines
Yes, makedumpfile needs to be linked against eppic library for filtering data and this will increase makedumpfile size and initramfs size too.
Just to clarify -- your example indicates that the vmlinux file is required for this facility to work, correct?
Quoted 1 line
makedumpfile -c -d 31 -x vmlinux --eppic
key.cvmcorefiltered_vmcoreClearly distros won't be putting the vmlinux file in the initramfs -- that's the whole reasoning behind vmcoreinfo. So the 99% of users that aren't interested in scrubbing will have to pay the penalty of the larger makedumpfile binary.
Quoted 6 lines
Quoted 2 lines
That's a good point Dave. We will never put debug compiled vmlinux in initramfs. Following two alternatives come to my mind.
As I mentioned, we don't need vmlinux in initramfs as filtering is done during post processing only.
You are missing the point. The point is that despite the fact that scrubbing will never be done from initramfs, all the users will pay penalty for increased makedumpfile size.
So why not write a separate tool (scrub-vmcore) so that makedumpfile users don't pay the bloated size penatly.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello,
Quoted 28 lines · On Fri, 16 Nov 2012 09:36:49 -0500 Vivek Goyal <[email protected]> wrote:
Quoted 20 lines · Aravinda Prasad
Quoted 16 lines · Vivek Goyal
On Thu, Nov 15, 2012 at 09:27:45AM -0500, Dave Anderson wrote:
[..]
Quoted 12 lines
Quoted 2 lines
Yes, makedumpfile needs to be linked against eppic library for filtering data and this will increase makedumpfile size and initramfs size too.
Just to clarify -- your example indicates that the vmlinux file is required for this facility to work, correct?
Quoted 1 line
makedumpfile -c -d 31 -x vmlinux --eppic
key.cvmcorefiltered_vmcoreClearly distros won't be putting the vmlinux file in the initramfs -- that's the whole reasoning behind vmcoreinfo. So the 99% of users that aren't interested in scrubbing will have to pay the penalty of the larger makedumpfile binary.
Quoted 6 lines
Quoted 2 lines
That's a good point Dave. We will never put debug compiled vmlinux in initramfs. Following two alternatives come to my mind.
As I mentioned, we don't need vmlinux in initramfs as filtering is done during post processing only.
Quoted 6 lines
You are missing the point. The point is that despite the fact that scrubbing will never be done from initramfs, all the users will pay penalty for increased makedumpfile size.
So why not write a separate tool (scrub-vmcore) so that makedumpfile users don't pay the bloated size penatly.
As Vivek said, I think it's difficult to persuade almost all distributor to specify EPPIC=on.
So, it seems that separate tool is better way for both user and distributor. If you will make it, I will drop eppic support from v1.5.1 and export core functions from makedumpfile as library for such tools in the future.
Does this meet your purpose, Aravinda ?
Thanks
Atsushi Kumagai
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
On 2012-11-20 15:17, Atsushi Kumagai wrote:
...
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines
As I mentioned, we don't need vmlinux in initramfs as filtering is done during post processing only.
You are missing the point. The point is that despite the fact that scrubbing will never be done from initramfs, all the users will pay penalty for increased makedumpfile size.
So why not write a separate tool (scrub-vmcore) so that makedumpfile users don't pay the bloated size penatly.
Quoted 8 lines
As Vivek said, I think it's difficult to persuade almost all distributor to specify EPPIC=on.
So, it seems that separate tool is better way for both user and distributor. If you will make it, I will drop eppic support from v1.5.1 and export core functions from makedumpfile as library for such tools in the future.
Does this meet your purpose, Aravinda ?
Vivek, Atsushi,
How about distributions building two targets one with default makedumpfile and other with EPPIC=on, which can be done by a simple modification to Makefile. Default makedumpfile will go into initramfs and the one with EPPIC=on can be used for filtering. This will avoid initramfs bloat and will also save effort in developing a new tool and exporting makedumpfile functions to a library. Also if we build a new tool, eventually, we will end up sharing the existing makdumpfile code.
Quoted 10 lines
Thanks
Atsushi Kumagai_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 26 lines · Aravinda Prasad
On 2012-11-20 15:17, Atsushi Kumagai wrote:
...
Quoted 19 lines
Quoted 11 lines
Quoted 4 lines
As I mentioned, we don't need vmlinux in initramfs as filtering is done during post processing only.
You are missing the point. The point is that despite the fact that scrubbing will never be done from initramfs, all the users will pay penalty for increased makedumpfile size.
So why not write a separate tool (scrub-vmcore) so that makedumpfile users don't pay the bloated size penatly.
As Vivek said, I think it's difficult to persuade almost all distributor to specify EPPIC=on.
So, it seems that separate tool is better way for both user and distributor. If you will make it, I will drop eppic support from v1.5.1 and export core functions from makedumpfile as library for such tools in the future.
Quoted 8 lines
Quoted 1 line
Does this meet your purpose, Aravinda ?
Vivek, Atsushi,
How about distributions building two targets one with default makedumpfile and other with EPPIC=on, which can be done by a simple modification to
Makefile.
That's not a good idea. Generally distributions don't ship multiple binaries with minor differences like that. I think going with a separate tool for doing this kind of filtering is better.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 32 lines · Vivek Goyal
Quoted 31 lines · Aravinda Prasad
On 2012-11-20 15:17, Atsushi Kumagai wrote:
...
Quoted 12 lines
Quoted 11 lines
Quoted 4 lines
As I mentioned, we don't need vmlinux in initramfs as filtering is done during post processing only.
You are missing the point. The point is that despite the fact that scrubbing will never be done from initramfs, all the users will pay penalty for increased makedumpfile size.
So why not write a separate tool (scrub-vmcore) so that makedumpfile users don't pay the bloated size penatly.
Quoted 8 lines
As Vivek said, I think it's difficult to persuade almost all distributor to specify EPPIC=on.
So, it seems that separate tool is better way for both user and distributor. If you will make it, I will drop eppic support from v1.5.1 and export core functions from makedumpfile as library for such tools in the future.
Does this meet your purpose, Aravinda ?
Vivek, Atsushi,
Quoted 7 lines
Quoted 3 lines
How about distributions building two targets one with default makedumpfile and other with EPPIC=on, which can be done by a simple modification to
Makefile.That's not a good idea. Generally distributions don't ship multiple binaries with minor differences like that. I think going with a separate tool for doing this kind of filtering is better.
Customers who want to filter their dump would certainly prefer a single tool and a single shot to throw away pages, scrub data and then compress, split or send the resulting dump to some target using ssh. This important functionality is lost if scrubbing is not part of makedumpfile and a separate tool is developed which just does scrubbing.
Quoted 9 lines
Thanks
Vivek_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
On Thu, Nov 22, 2012 at 10:44:49PM +0530, Aravinda Prasad wrote:
[..]
Quoted 5 lines
Customers who want to filter their dump would certainly prefer a single tool and a single shot to throw away pages, scrub data and then compress, split or send the resulting dump to some target using ssh. This important functionality is lost if scrubbing is not part of makedumpfile and a separate tool is developed which just does scrubbing.
Customers do filtering from initramfs context. It is not practical to save TB of memory and filter it later. And we just concluded that is it not practical to do scrubbing from initramfs due to need of vmlinux.
So doing filtering and doing scrubbing will not happen at the same time practically. So bloating size of makedumpfile does not make sense to me.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 9 lines · Vivek Goyal
On Thu, Nov 22, 2012 at 10:44:49PM +0530, Aravinda Prasad wrote:
[..]
Quoted 5 lines
Customers who want to filter their dump would certainly prefer a single tool and a single shot to throw away pages, scrub data and then compress, split or send the resulting dump to some target using ssh. This important functionality is lost if scrubbing is not part of makedumpfile and a separate tool is developed which just does scrubbing.
Quoted 8 lines
Customers do filtering from initramfs context. It is not practical to save TB of memory and filter it later. And we just concluded that is it not practical to do scrubbing from initramfs due to need of vmlinux.
So doing filtering and doing scrubbing will not happen at the same time practically. So bloating size of makedumpfile does not make sense to me.
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to makedumpfile code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building .so will be similar to what we have in crash.
Quoted 3 lines
Thanks
Vivek
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 23 lines · Aravinda Prasad
Quoted 9 lines · Vivek Goyal
On Thu, Nov 22, 2012 at 10:44:49PM +0530, Aravinda Prasad wrote:
[..]
Quoted 5 lines
Customers who want to filter their dump would certainly prefer a single tool and a single shot to throw away pages, scrub data and then compress, split or send the resulting dump to some target using ssh. This important functionality is lost if scrubbing is not part of makedumpfile and a separate tool is developed which just does scrubbing.
Quoted 8 lines
Customers do filtering from initramfs context. It is not practical to save TB of memory and filter it later. And we just concluded that is it not practical to do scrubbing from initramfs due to need of vmlinux.
So doing filtering and doing scrubbing will not happen at the same time practically. So bloating size of makedumpfile does not make sense to me.
Quoted 8 lines
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to makedumpfile code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building .so will be similar to what we have in crash.
It will still show up in dynamic library dependencing using ldd. We will have to put some hack to exclude the leppic despite the fact that makedumpfile is dependent on it.
In F18, now we use dracut to build kdump initramfs. On command line we specify any extra binaries to be included and makedumpfile is one of those. Dracut will determine all the dependencies and automatically pull these in. So even if we don't use --eppic flag, dracult will pull in eppic shared library anyway.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 37 lines · Vivek Goyal
Quoted 23 lines · Aravinda Prasad
Quoted 9 lines · Vivek Goyal
On Thu, Nov 22, 2012 at 10:44:49PM +0530, Aravinda Prasad wrote:
[..]
Quoted 5 lines
Customers who want to filter their dump would certainly prefer a single tool and a single shot to throw away pages, scrub data and then compress, split or send the resulting dump to some target using ssh. This important functionality is lost if scrubbing is not part of makedumpfile and a separate tool is developed which just does scrubbing.
Quoted 8 lines
Customers do filtering from initramfs context. It is not practical to save TB of memory and filter it later. And we just concluded that is it not practical to do scrubbing from initramfs due to need of vmlinux.
So doing filtering and doing scrubbing will not happen at the same time practically. So bloating size of makedumpfile does not make sense to me.
Quoted 8 lines
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to makedumpfile code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building .so will be similar to what we have in crash.
It will still show up in dynamic library dependencing using ldd. We will have to put some hack to exclude the leppic despite the fact that makedumpfile is dependent on it.
Quoted 5 lines
In F18, now we use dracut to build kdump initramfs. On command line we specify any extra binaries to be included and makedumpfile is one of those. Dracut will determine all the dependencies and automatically pull these in. So even if we don't use --eppic flag, dracult will pull in eppic shared library anyway.
I think ldd or dracut will not be able to make out that we are dependent on libeppic.so as we will be using the dlopen("libeppic.so, ...) call to dynamically load. No extra flags will be added to Makefile to specify -leppic while building makedumpfile. Hence, from my understanding, while building kdump initramfs, dracut cannot determine that we are dependent on eppic
Quoted 4 lines
Thanks
Vivek
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
On Mon, Dec 03, 2012 at 08:05:54PM +0530, Aravinda Prasad wrote:
[..]
Quoted 20 lines
Quoted 13 lines
Quoted 8 lines
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to makedumpfile code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building .so will be similar to what we have in crash.
It will still show up in dynamic library dependencing using ldd. We will have to put some hack to exclude the leppic despite the fact that makedumpfile is dependent on it.
Quoted 5 lines
In F18, now we use dracut to build kdump initramfs. On command line we specify any extra binaries to be included and makedumpfile is one of those. Dracut will determine all the dependencies and automatically pull these in. So even if we don't use --eppic flag, dracult will pull in eppic shared library anyway.
Quoted 6 lines
I think ldd or dracut will not be able to make out that we are dependent on libeppic.so as we will be using the
dlopen("libeppic.so, ...)call to dynamically load. No extra flags will be added toMakefileto specify -leppic while building makedumpfile. Hence, from my understanding, while building kdump initramfs, dracut cannot determine that we are dependent on eppic
Ok, if dracut does not pull in libeppic and does not bloat size of initramfs, I am fine.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello Aravinda,
Quoted 30 lines · On Mon, 3 Dec 2012 13:40:01 -0500 Vivek Goyal <[email protected]> wrote:
On Mon, Dec 03, 2012 at 08:05:54PM +0530, Aravinda Prasad wrote:
[..]
Quoted 20 lines
Quoted 13 lines
Quoted 8 lines
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to makedumpfile code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building .so will be similar to what we have in crash.
It will still show up in dynamic library dependencing using ldd. We will have to put some hack to exclude the leppic despite the fact that makedumpfile is dependent on it.
Quoted 5 lines
In F18, now we use dracut to build kdump initramfs. On command line we specify any extra binaries to be included and makedumpfile is one of those. Dracut will determine all the dependencies and automatically pull these in. So even if we don't use --eppic flag, dracult will pull in eppic shared library anyway.
Quoted 6 lines
I think ldd or dracut will not be able to make out that we are dependent on libeppic.so as we will be using the
dlopen("libeppic.so, ...)call to dynamically load. No extra flags will be added toMakefileto specify -leppic while building makedumpfile. Hence, from my understanding, while building kdump initramfs, dracut cannot determine that we are dependent on eppic
Quoted 2 lines
Ok, if dracut does not pull in libeppic and does not bloat size of initramfs, I am fine.
I agree with this idea too.
However, the release date is closing in, so would you re-send the patch set based on v1.5.1 ? I will accept it as official feature in v1.5.2.
Thanks
Atsushi Kumagai
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 35 lines · Atsushi Kumagai
Hello Aravinda,
Quoted 30 lines · On Mon, 3 Dec 2012 13:40:01 -0500 Vivek Goyal <[email protected]> wrote:
On Mon, Dec 03, 2012 at 08:05:54PM +0530, Aravinda Prasad wrote:
[..]
Quoted 26 lines
Quoted 13 lines
Quoted 8 lines
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to makedumpfile code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building .so will be similar to what we have in crash.
It will still show up in dynamic library dependencing using ldd. We will have to put some hack to exclude the leppic despite the fact that makedumpfile is dependent on it.
Quoted 5 lines
In F18, now we use dracut to build kdump initramfs. On command line we specify any extra binaries to be included and makedumpfile is one of those. Dracut will determine all the dependencies and automatically pull these in. So even if we don't use --eppic flag, dracult will pull in eppic shared library anyway.
I think ldd or dracut will not be able to make out that we are dependent on libeppic.so as we will be using the
dlopen("libeppic.so, ...)call to dynamically load. No extra flags will be added toMakefileto specify -leppic while building makedumpfile. Hence, from my understanding, while building kdump initramfs, dracut cannot determine that we are dependent on eppic
Quoted 8 lines
Quoted 2 lines
Ok, if dracut does not pull in libeppic and does not bloat size of initramfs, I am fine.
I agree with this idea too.
However, the release date is closing in, so would you re-send the patch set based on v1.5.1 ? I will accept it as official feature in v1.5.2.
sure. Thanks Atsushi
Quoted 4 lines
Thanks
Atsushi Kumagai
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
----- Original Message -----
Quoted 10 lines
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to makedumpfile code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building .so will be similar to what we have in crash.
OK, so based upon the most recent discussion here:
Re: [PATCH v2 0/7] makedumpfile security key filtering with eppic
http://lists.infradead.org/pipermail/kexec/2012-December/007452.html
Aravinda states that he will be posting an update for makedumpfile that will dlopen() a new "libeppic.so" shared object that is built with the libeppic.a library.
The upstream eppic git tree contains an "applications/crash" subdirectory that contains the code that creates a crash-utility-specific "eppic.so" file.
Accordingly, this new proposal will also require new source file(s) and a build process to create this new makedumpfile-specific "libeppic.so" shared object.
Arvinda -- will you be proposing an additional "applications/makedumpfile" subdirectory for the upstream eppic git tree that can be used to create the makedumpfile-specific libeppic.so file?
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 27 lines · Dave Anderson
----- Original Message -----
Quoted 10 lines
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to makedumpfile code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building .so will be similar to what we have in crash.
OK, so based upon the most recent discussion here:
Re: [PATCH v2 0/7] makedumpfile security key filtering with eppic
http://lists.infradead.org/pipermail/kexec/2012-December/007452.htmlAravinda states that he will be posting an update for makedumpfile that will
dlopen()a new "libeppic.so" shared object that is built with the libeppic.a library.The upstream eppic git tree contains an "applications/crash" subdirectory that contains the code that creates a crash-utility-specific "eppic.so" file.
Quoted 7 lines
Accordingly, this new proposal will also require new source file(s) and a build process to create this new makedumpfile-specific "libeppic.so" shared object.
Arvinda -- will you be proposing an additional "applications/makedumpfile" subdirectory for the upstream eppic git tree that can be used to create the makedumpfile-specific libeppic.so file?
This is what I am planning.
A new extension_eppic.c file will be created under makedumpfile source directory. This file is equivalent to applications/crash/eppic.c in upstream eppic repository. A new target will be added to the Makefile of makedumpfile to build the shared library and to build this shared library libeppic.a would be required. The makedumpfile specific shared library will be named different - say eppic_mkdumpfile.so to avoid conflict with crash specific libeppic.so.
The reason for including extension_eppic.c under makedumpfile source is because it will be dependent on other functions in makedumpfile code like dwarf related calls etc. People modifying those functions should be aware of the callers in extension_eppic.c and if this is included in upstream eppic code, it will be easily overlooked (or may not be aware of its existence).
A flag --eppic will be added to makedumpfile which will dlopen the shared library. Few additional helper functions will be added to dwarf_info.c and erase_info.c - the same functions which are in v2 of the patchset.
Quoted 9 lines
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 10 lines
-----Original Message-----
From: Aravinda Prasad [mailto:[email protected]]
Sent: Friday, December 07, 2012 1:05 AM
To: Dave Anderson
Cc: [email protected]; [email protected]; [email protected]; Luc Chouinard; [email protected]; Atsushi Kumagai; Vivek Goyal; [email protected]
Subject: Re: [PATCH v2 0/7] makedumpfile security key filtering with
eppic
Quoted 12 lines · Dave Anderson
Quoted 8 lines
----- Original Message -----
Quoted 3 lines
Another approach is to dynamically load libeppic library - similar way how crash does it. No major changes will be done to
makedumpfile
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
code, except the addition of --eppic flag. Upon specifying --eppic flag makedumpfile will dlopen libeppic.so, which will have functionality to scrub the specified data. This will prevent makedumpfile bloat and will not affect the size of initramfs as --eppic is only specified during post processing. The distribution should build and ship libeppic.so and the procedure for building
.so
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
will be similar to what we have in crash.
OK, so based upon the most recent discussion here:
Re: [PATCH v2 0/7] makedumpfile security key filtering with eppic
http://lists.infradead.org/pipermail/kexec/2012-December/007452.html
Quoted 6 lines
Quoted 6 lines
Aravinda states that he will be posting an update for makedumpfile that will
dlopen()a new "libeppic.so" shared object that is built with the libeppic.a library.The upstream eppic git tree contains an "applications/crash" subdirectory that contains the code that creates a
crash-utility-specific
Quoted 4 lines
"eppic.so" file.
Quoted 3 lines
Accordingly, this new proposal will also require new source file(s) and a build process to create this new makedumpfile-specific
"libeppic.so"
Quoted 3 lines
Quoted 3 lines
shared object.
Arvinda -- will you be proposing an additional
"applications/makedumpfile"
Quoted 8 lines
Quoted 2 lines
subdirectory for the upstream eppic git tree that can be used to create the makedumpfile-specific libeppic.so file?
This is what I am planning.
A new
extension_eppic.cfile will be created under makedumpfile source directory. This file is equivalent toapplications/crash/eppic.cin
upstream eppic
Quoted 1 line
repository. A new target will be added to the
Makefileof makedumpfile
to build
Quoted 1 line
the shared library and to build this shared library libeppic.a would
be required.
Quoted 4 lines
The makedumpfile specific shared library will be named different - say eppic_mkdumpfile.so to avoid conflict with crash specific libeppic.so.
The reason for including
extension_eppic.cunder makedumpfile source
is
Quoted 1 line
because it will be dependent on other functions in makedumpfile code
like dwarf
Quoted 1 line
related calls etc. People modifying those functions should be aware of
the
Quoted 1 line
callers in
extension_eppic.cand if this is included in upstream eppic
code, it will
Quoted 1 line
be easily overlooked (or may not be aware of its existence).
The same reasons exists for applications/crash/eppic.c. It only ended up on eppic's git for two reasons. To provide for a complete example of integration into a tool and, because I originally wrote it, I could support it from there without having to send a patch Dave's way.
I expect other applications to own their glue/integration file(s) and only grab libeppic from the git. These applications, will comply to libeppic's APIs and only fixes or new functions will be added to libeppic to support these applications and valid new requirements they may have. Like we did for mkdumpfile.
Same libeppic.so and same libeppic Makefile for all.
Quoted 2 lines
A flag --eppic will be added to makedumpfile which will dlopen the
shared
Quoted 1 line
library. Few additional helper functions will be added to
dwarf_info.c
and
Quoted 14 lines
erase_info.c- the same functions which are in v2 of the patchset.Quoted 9 lines
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 3 lines
--
Regards,
Aravinda
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
On Fri, Dec 07, 2012 at 08:46:33AM -0500, Luc Chouinard wrote:
[..]
Quoted 27 lines
Quoted 4 lines
This is what I am planning.
A new
extension_eppic.cfile will be created under makedumpfile source directory. This file is equivalent toapplications/crash/eppic.cinupstream eppic
Quoted 1 line
repository. A new target will be added to the
Makefileof makedumpfileto build
Quoted 1 line
the shared library and to build this shared library libeppic.a would
be required.
Quoted 4 lines
The makedumpfile specific shared library will be named different - say eppic_mkdumpfile.so to avoid conflict with crash specific libeppic.so.
The reason for including
extension_eppic.cunder makedumpfile sourceis
Quoted 1 line
because it will be dependent on other functions in makedumpfile code
like dwarf
Quoted 1 line
related calls etc. People modifying those functions should be aware of
the
Quoted 1 line
callers in
extension_eppic.cand if this is included in upstream eppiccode, it will
Quoted 1 line
be easily overlooked (or may not be aware of its existence).
The same reasons exists for
applications/crash/eppic.c. It only ended up on eppic's git for two reasons. To provide for a complete example of integration into a tool and, because I originally wrote it, I could support it from there without having to send a patch Dave's way.
Quoted 7 lines
I expect other applications to own their glue/integration file(s) and only grab libeppic from the git. These applications, will comply to libeppic's APIs and only fixes or new functions will be added to libeppic to support these applications and valid new requirements they may have. Like we did for mkdumpfile.
Same libeppic.so and same libeppic
Makefilefor all.
I am lost here. Can somebody explain this thing in layman's terms with simple example. (We have a function foo() in libeppic which makeudmpfile wants to call. Now what?)
Few questions come to my mind.
- What is glue logic and why do we need application specific glue logic to use this library.
- Assuming we need glue logic and assuming it is small enough, will it make sense to build statically build glue logic in application and build actual libeppic as shared object and do dlopen() on it.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 38 lines · Vivek Goyal
On Fri, Dec 07, 2012 at 08:46:33AM -0500, Luc Chouinard wrote:
[..]
Quoted 27 lines
Quoted 4 lines
This is what I am planning.
A new
extension_eppic.cfile will be created under makedumpfile source directory. This file is equivalent toapplications/crash/eppic.cinupstream eppic
Quoted 1 line
repository. A new target will be added to the
Makefileof makedumpfileto build
Quoted 1 line
the shared library and to build this shared library libeppic.a would
be required.
Quoted 4 lines
The makedumpfile specific shared library will be named different - say eppic_mkdumpfile.so to avoid conflict with crash specific libeppic.so.
The reason for including
extension_eppic.cunder makedumpfile sourceis
Quoted 1 line
because it will be dependent on other functions in makedumpfile code
like dwarf
Quoted 1 line
related calls etc. People modifying those functions should be aware of
the
Quoted 1 line
callers in
extension_eppic.cand if this is included in upstream eppiccode, it will
Quoted 1 line
be easily overlooked (or may not be aware of its existence).
The same reasons exists for
applications/crash/eppic.c. It only ended up on eppic's git for two reasons. To provide for a complete example of integration into a tool and, because I originally wrote it, I could support it from there without having to send a patch Dave's way.Quoted 7 lines
I expect other applications to own their glue/integration file(s) and only grab libeppic from the git. These applications, will comply to libeppic's APIs and only fixes or new functions will be added to libeppic to support these applications and valid new requirements they may have. Like we did for mkdumpfile.
Same libeppic.so and same libeppic
Makefilefor all.
Quoted 8 lines
I am lost here. Can somebody explain this thing in layman's terms with simple example. (We have a function
foo()in libeppic which makeudmpfile wants to call. Now what?)Few questions come to my mind.
- What is glue logic and why do we need application specific glue logic to use this library.
Consider this simple eppic macro, which prints the value of the global kernel variable "modules":
smod()
{
struct list_head *mod;
mod = (struct list_head *)modules;
printf("Modules = %p \n", mod);
}Eppic has logic to interpret this C program, but needs some help in resolving the actual value of "modules" (as well as members of struct list_head) and defines few call-back functions to help it. This generic logic forms libeppic.a. The glue logic implements these call back function to help libeppic.a.
Eg: whenever libeppic.a encounters struct list_head, it calls apimember("list_head") call back function to learn about the details of the members of the structure. Glue logic implement this call-back function and calls libeppic.a calls with information - eppic_member_sname("list_head", "next"), eppic_member_sname("list_head", "prev") etc., inside apimember("list_head").
In crash, the glue logic contains gdb related calls and in makedumpfile it contains dwarf related calls to get the structure members, value of global variables etc. However, libeppic.a does not care what calls we use in the background. It just needs the data. This makes it very generic and powerful.
Now the glue logic along with libeppic.a makes the shared library *.so (named eppic.so for crash, planning to name eppic_makedumpfile.so for makedumpfile), which applications can dlopen().
Hope this helps.
Quoted 5 lines
- Assuming we need glue logic and assuming it is small enough, will it make sense to build statically build glue logic in application and build actual libeppic as shared object and do
dlopen()on it.
Glue logic is 450K lines of code (all of them in extension_eppic.c) and calls lot of eppic library calls, making this glue logic static requires libeppic.a to be statically included, which was our first approach.
Quoted 3 lines
Thanks
Vivek
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Quoted 89 lines · Aravinda Prasad
Quoted 38 lines · Vivek Goyal
On Fri, Dec 07, 2012 at 08:46:33AM -0500, Luc Chouinard wrote:
[..]
Quoted 27 lines
Quoted 4 lines
This is what I am planning.
A new
extension_eppic.cfile will be created under makedumpfile source directory. This file is equivalent toapplications/crash/eppic.cinupstream eppic
Quoted 1 line
repository. A new target will be added to the
Makefileof makedumpfileto build
Quoted 1 line
the shared library and to build this shared library libeppic.a would
be required.
Quoted 4 lines
The makedumpfile specific shared library will be named different - say eppic_mkdumpfile.so to avoid conflict with crash specific libeppic.so.
The reason for including
extension_eppic.cunder makedumpfile sourceis
Quoted 1 line
because it will be dependent on other functions in makedumpfile code
like dwarf
Quoted 1 line
related calls etc. People modifying those functions should be aware of
the
Quoted 1 line
callers in
extension_eppic.cand if this is included in upstream eppiccode, it will
Quoted 1 line
be easily overlooked (or may not be aware of its existence).
The same reasons exists for
applications/crash/eppic.c. It only ended up on eppic's git for two reasons. To provide for a complete example of integration into a tool and, because I originally wrote it, I could support it from there without having to send a patch Dave's way.Quoted 7 lines
I expect other applications to own their glue/integration file(s) and only grab libeppic from the git. These applications, will comply to libeppic's APIs and only fixes or new functions will be added to libeppic to support these applications and valid new requirements they may have. Like we did for mkdumpfile.
Same libeppic.so and same libeppic
Makefilefor all.Quoted 8 lines
I am lost here. Can somebody explain this thing in layman's terms with simple example. (We have a function
foo()in libeppic which makeudmpfile wants to call. Now what?)Few questions come to my mind.
- What is glue logic and why do we need application specific glue logic to use this library.
Consider this simple eppic macro, which prints the value of the global kernel variable "modules":
smod(){ struct list_head *mod; mod = (struct list_head *)modules; printf("Modules = %p \n", mod); }Eppic has logic to interpret this C program, but needs some help in resolving the actual value of "modules" (as well as members of struct
list_head) and defines few call-back functions to help it. This generic logic forms libeppic.a. The glue logic implements these call back function to help libeppic.a.Eg: whenever libeppic.a encounters struct
list_head, it callsapimember("list_head")call back function to learn about the details of the members of the structure. Glue logic implement this call-back function and calls libeppic.a calls with information -eppic_member_sname("list_head", "next"),eppic_member_sname("list_head", "prev")etc., insideapimember("list_head").In crash, the glue logic contains gdb related calls and in makedumpfile it contains dwarf related calls to get the structure members, value of global variables etc. However, libeppic.a does not care what calls we use in the background. It just needs the data. This makes it very generic and powerful.
Now the glue logic along with libeppic.a makes the shared library *.so (named eppic.so for crash, planning to name eppic_makedumpfile.so for makedumpfile), which applications can
dlopen().Hope this helps.
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
- Assuming we need glue logic and assuming it is small enough, will it make sense to build statically build glue logic in application and build actual libeppic as shared object and do
dlopen()on it.Glue logic is 450K lines of code (all of them in
extension_eppic.c) and
Correction: it is just 450 lines of code not 450K !!
Quoted 9 lines
calls lot of eppic library calls, making this glue logic static requires libeppic.a to be statically included, which was our first approach.
Quoted 3 lines
Thanks
Vivek
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
On 2012-11-15 19:57, Dave Anderson wrote:
...
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
Again, how distributions will handle it. If it is being integrated makedumpfile, as opposed to an stand alone utility, that means it makedumpfile needs to link against this library so that somebody can later filter out the symbols. And that means initramfs size bloat too?
Yes, makedumpfile needs to be linked against eppic library for filtering data and this will increase makedumpfile size and initramfs size too.
Just to clarify -- your example indicates that the vmlinux file is required for this facility to work, correct?
Yes vmlinux file is required
Quoted 7 lines
Quoted 2 lines
makedumpfile -c -d 31 -x vmlinux --eppic
key.cvmcorefiltered_vmcoreClearly distros won't be putting the vmlinux file in the initramfs -- that's the whole reasoning behind vmcoreinfo. So the 99% of users that aren't interested in scrubbing will have to pay the penalty of the larger makedumpfile binary.
vmlinux is not required in intitramfs as the filtering is done during post processing i.e., after the the dump ia saved
Quoted 3 lines
Dave
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
On Thu, Nov 15, 2012 at 06:20:44PM +0530, Aravinda Prasad wrote:
[..]
Quoted 44 lines
Quoted 2 lines
Can you give some details about how does it work and what's the correlation with makedumpfile.
struct key in
include/linux/key.hholds "authentication token"/"access credential"/"keyring". Suppose these entries should be scrubbed from the dumpfile. Then thekeyring_name_hashhash table should be scanned and for each non-empty list, the entire list should be traversed and payload.value (or any other data) in struct key should be cleared.Now the EPPIC macro looks like this:
int
key(){ int i; struct list_head *head; struct list_head *next, *prev; head = (struct list_head *)keyring_name_hash; for (i = 0; i < 32; i++) { next = (struct list_head *) head[i].next; prev = (struct list_head *) head[i].prev; if (!next) continue; do { struct key *mykey, *off = 0; mykey = (struct key *)((unsigned long)(next) - ((unsigned long)&(off->type_data))); memset((char *)mykey->payload.value, 'X', 0x8); next = *(struct list_head **) mykey->type_data.link.next; } while (next != prev); } return 1; }
Quoted 5 lines
The data can be cleared by specifying:
makedumpfile -c -d 31 -x vmlinux --eppickey.cvmcorefiltered_vmcoremakedumpfile with the help of eppic will interpret the macro
key.c, traverses all the hash chains and erases paylod.value of struct key.
Ok, are these the only places where key is. Can a copy of it exist in some other buffers? We don't clear these.
Also, if key is the only issue, why not just write this logic in makedumpfile and provide another option, --clear-kernel-keys.
Why to introduce such generic scheme.
[..]
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
Memory footprint of libeppic.a is around 1MB. Yes, this will be statically linked to makedumpfile. Users should specify EPPIC=on while building the makedumpfile and hence linking libeppic.a is optional
1MB bloat is significant given the fact that we reserve only 128MB for kdump kernel. Hence we need to review the benefits of this bloat very carefully.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 47 lines · Vivek Goyal
On Thu, Nov 15, 2012 at 06:20:44PM +0530, Aravinda Prasad wrote:
[..]
Quoted 44 lines
Quoted 2 lines
Can you give some details about how does it work and what's the correlation with makedumpfile.
struct key in
include/linux/key.hholds "authentication token"/"access credential"/"keyring". Suppose these entries should be scrubbed from the dumpfile. Then thekeyring_name_hashhash table should be scanned and for each non-empty list, the entire list should be traversed and payload.value (or any other data) in struct key should be cleared.Now the EPPIC macro looks like this:
int
key(){ int i; struct list_head *head; struct list_head *next, *prev; head = (struct list_head *)keyring_name_hash; for (i = 0; i < 32; i++) { next = (struct list_head *) head[i].next; prev = (struct list_head *) head[i].prev; if (!next) continue; do { struct key *mykey, *off = 0; mykey = (struct key *)((unsigned long)(next) - ((unsigned long)&(off->type_data))); memset((char *)mykey->payload.value, 'X', 0x8); next = *(struct list_head **) mykey->type_data.link.next; } while (next != prev); } return 1; }
Quoted 8 lines
Quoted 5 lines
The data can be cleared by specifying:
makedumpfile -c -d 31 -x vmlinux --eppickey.cvmcorefiltered_vmcoremakedumpfile with the help of eppic will interpret the macro
key.c, traverses all the hash chains and erases paylod.value of struct key.Ok, are these the only places where key is. Can a copy of it exist in some other buffers? We don't clear these.
I don't think a copy exist in other places
Quoted 5 lines
Also, if key is the only issue, why not just write this logic in makedumpfile and provide another option, --clear-kernel-keys.
Why to introduce such generic scheme.
key is not the only issue, it was just an example. There could be other things as well (data in socket buffers, device driver buffers, etc) which customers may consider sensitive/private and are interested in scrubbing.
Also this is an extension to the already existing generic solution implemented in makedumpfile, where rules can be specified using --config option. This extension is built on the existing infrastructure and provides a more flexible and powerful way to specify the data to be scrubbed. For eg, scrubbing the keyring data mentioned in one of my previous mails would not be possible with --config option.
Quoted 18 lines
[..]
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
Memory footprint of libeppic.a is around 1MB. Yes, this will be statically linked to makedumpfile. Users should specify EPPIC=on while building the makedumpfile and hence linking libeppic.a is optional
1MB bloat is significant given the fact that we reserve only 128MB for kdump kernel. Hence we need to review the benefits of this bloat very carefully.
Thanks
Vivek
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
On Fri, Nov 16, 2012 at 04:40:47PM +0530, Aravinda Prasad wrote:
[..]
Quoted 5 lines
Quoted 2 lines
Ok, are these the only places where key is. Can a copy of it exist in some other buffers? We don't clear these.
I don't think a copy exist in other places
I am wondering how does ssh work. User's private key is stored in .ssh/ and when authentication with server is happening then we must be signing something with that private key and most likely it will be in some buffer somewhere (user space buffer).
Quoted 13 lines
Quoted 6 lines
Also, if key is the only issue, why not just write this logic in makedumpfile and provide another option, --clear-kernel-keys.
Why to introduce such generic scheme.
key is not the only issue, it was just an example. There could be other things as well (data in socket buffers, device driver buffers, etc) which customers may consider sensitive/private and are interested in scrubbing.
Quoted 6 lines
Also this is an extension to the already existing generic solution implemented in makedumpfile, where rules can be specified using --config option. This extension is built on the existing infrastructure and provides a more flexible and powerful way to specify the data to be scrubbed. For eg, scrubbing the keyring data mentioned in one of my previous mails would not be possible with --config option.
I am not against building infrastructure to scrub vmcore. I am only concerned about size bloat of makedumpfile.
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
On Wed, Nov 14, 2012 at 10:36:53PM +0530, Aravinda Prasad wrote:
[..]
Quoted 3 lines
The main intention is to remove confidential information from the dump file, like ssh keys etc., which could be just few bytes, hence, may not reduce the size of the dump significantly.
Where are these ssh keys? Are these in user space memory or kernel memory? How would one know the location of these keys?
Thanks
Vivek
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 9 lines · Vivek Goyal
On Wed, Nov 14, 2012 at 10:36:53PM +0530, Aravinda Prasad wrote:
[..]
Quoted 3 lines
The main intention is to remove confidential information from the dump file, like ssh keys etc., which could be just few bytes, hence, may not reduce the size of the dump significantly.
Where are these ssh keys? Are these in user space memory or kernel memory? How would one know the location of these keys?
Check out my previous mail which explains the keyring scenario in detail. ssh key was a bad example.
Quoted 9 lines
Thanks
Vivek_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
----- Original Message -----
Quoted 35 lines
Hi Vivek,
Quoted 10 lines · Vivek Goyal
Quoted 8 lines · Aravinda Prasad
makedumpfile security key filtering enhancement - Add Eppic language support (formerly known as SIAL) to specify rules to scrub data in a dumpfile. Eppic was previously part of crash source code repository.
The following series of patches enhance the makedumpfile to provide a more powerful way to specify rules and commands to traverse and erase complex data structures in a dump file by integrating Embeddable Pre-Processor and Interpreter for C (eppic).
Quoted 5 lines
Hi Aravinda,
Had few questions.
- Which file will contain all the rules?
As of now rule files will not be provided by makedumpfile. However, writing a rule file is very easy - it is a C program.
Quoted 4 lines
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
Memory footprint of libeppic.a is around 1MB. Yes, this will be statically linked to makedumpfile. Users should specify EPPIC=on while building the makedumpfile and hence linking libeppic.a is optional
Quoted 1 line
Quoted 5 lines
Quoted 2 lines
- Is this supposed to work from kdump initramfs or it is supposed to be used on already saved dump (later during post processing).
For the time being, it is only during post processing.
By post-processing, I understand you to say that the system would be configured to do a full ELF vmcore dump, save it somewhere, and then somebody would do the post-processing at a later time?
Or is it possible to run makedumpfile again on a compressed kdump that was previously created at dump-time?
Quoted 9 lines
Quoted 4 lines
Given the fact that it does not reduce the size of core file significantly, I would think that it is better to post process vmcore to wipe out some symbols.
The main intention is to remove confidential information from the dump file, like ssh keys etc., which could be just few bytes, hence, may not reduce the size of the dump significantly.
So this would require you to first do a crash analysis on the unfiltered dumpfile, find out what you want to filter, write a script, and then run makedumpfile on the vmcore, correct?
Dave
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 42 lines · Dave Anderson
----- Original Message -----
Quoted 39 lines
Hi Vivek,
Quoted 10 lines · Vivek Goyal
Quoted 8 lines · Aravinda Prasad
makedumpfile security key filtering enhancement - Add Eppic language support (formerly known as SIAL) to specify rules to scrub data in a dumpfile. Eppic was previously part of crash source code repository.
The following series of patches enhance the makedumpfile to provide a more powerful way to specify rules and commands to traverse and erase complex data structures in a dump file by integrating Embeddable Pre-Processor and Interpreter for C (eppic).
Quoted 5 lines
Hi Aravinda,
Had few questions.
- Which file will contain all the rules?
As of now rule files will not be provided by makedumpfile. However, writing a rule file is very easy - it is a C program.
Quoted 4 lines
- What's the memory footprint of libeppic.a? Looks like this will be linked statically with makedumpfile, and how much is the size bloat of makedumpfile.
Memory footprint of libeppic.a is around 1MB. Yes, this will be statically linked to makedumpfile. Users should specify EPPIC=on while building the makedumpfile and hence linking libeppic.a is optional
Quoted 3 lines
- Is this supposed to work from kdump initramfs or it is supposed to be used on already saved dump (later during post processing).
Quoted 5 lines
Quoted 1 line
For the time being, it is only during post processing.
By post-processing, I understand you to say that the system would be configured to do a full ELF vmcore dump, save it somewhere, and then somebody would do the post-processing at a later time?
Yes exactly.
Quoted 3 lines
Or is it possible to run makedumpfile again on a compressed kdump that was previously created at dump-time?
Yes, it is possible to run makedumpfile with eppic again and again on compressed kdump, dump which excludes unnecessary pages and dumpfile which is already filtered by eppic.
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Given the fact that it does not reduce the size of core file significantly, I would think that it is better to post process vmcore to wipe out some symbols.
Quoted 7 lines
Quoted 3 lines
The main intention is to remove confidential information from the dump file, like ssh keys etc., which could be just few bytes, hence, may not reduce the size of the dump significantly.
So this would require you to first do a crash analysis on the unfiltered dumpfile, find out what you want to filter, write a script, and then run makedumpfile on the vmcore, correct?
Need not be always. If users know what to erase in advance they can skip the crash analysis part. I had mentioned an example in my previous reply to Vivek on how to erase keyring data from struct key.
Quoted 4 lines
Dave
Signature · 8 lines
Regards, Aravinda _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec