chunkd: don't leak an FS object iterator

chk_list_objs called fs_list_objs_open without also calling fs_list_objs_close.

32,808 bytes in 1 blocks are definitely lost in loss record 413 of 419

    at 0x4A0515D: malloc (vg_replace_malloc.c:195)
    by 0x31BA8A26D0: __alloc_dir (opendir.c:184)
    by 0x405619: fs_list_objs_open (be-fs.c:974)
    by 0x40B202: chk_list_objs (selfcheck.c:41)
    by 0x40B575: chk_dbscan (selfcheck.c:131)
    by 0x40B628: chk_thread_scan (selfcheck.c:147)
    by 0x40B757: chk_thread_command (selfcheck.c:179)
    by 0x40B890: chk_thread_func (selfcheck.c:219)
    by 0x31BC464E83: g_thread_create_proxy (gthread.c:1893)
    by 0x31BB407760: start_thread (pthread_create.c:301)
    by 0x31BA8E151C: clone (clone.S:115)

---

Thanks to Pete for catching my error.
To make up for that, here's a real leak fix:

 chunkd/selfcheck.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

index f3713da..86d3eb2 100644
@@ -100,6 +100,7 @@ static void chk_list_objs(struct chk_tls *tls, uint32_t table_id)
 
 		free(fn);
 	}
+	fs_list_objs_close(&lister);
 }
 
 static void chk_dbscan(struct chk_tls *tls)

--
1.7.3.293.gca9a76

After seeing a few valgrind references from you, I'm curious... do you by chance happen to have a valgrind suppression file for openssl on Fedora?

I've been wanting to run valgrind on chunkd, but each time I attempt it, I -- and valgrind -- have been overwhelmed by openssl false positives. openssl, deep in its RAND_xxx functions, intentionally does crazy stuff like using random, uninitialized stack contents as RNG entropy. Cute, but valgrind quite rightly complains loudly about it.

It's a topic I've been meaning to research, because I currently lack the valgrind-fu necessary to have an effective valgrind+chunkd session.

Thanks,

Jeff

No, I ignored them. But to create a so-called "suppressions" file, you can invoke valgrind with --gen-suppressions=all. You select the ssl-related entries and put them in a file, say F, then whenever you run valgrind, use --suppressions=F to make it ignore those.

BTW, Here are a few more that I haven't investigated:

==4506== 1 errors in context 1 of 7:
==4506== Syscall param pwrite64(buf) points to uninitialised byte(s)

==4506==    at 0x31BB40EDE3: __pwrite_nocancel (syscall-template.S:82)
==4506==    by 0x31BE137AD0: __os_io (os_rw.c:92)
==4506==    by 0x31BE125733: __memp_pgwrite (mp_bh.c:394)
==4506==    by 0x31BE125954: __memp_bhwrite (mp_bh.c:168)
==4506==    by 0x31BE134482: __memp_sync_int (mp_sync.c:530)
==4506==    by 0x31BE0CBD66: __db_sync (db_am.c:706)
==4506==    by 0x31BE0CA652: __db_refresh (db.c:820)
==4506==    by 0x31BE0CA8C0: __db_close (db.c:695)
==4506==    by 0x31BE0E64A7: __db_close_pp (db_iface.c:253)
==4506==    by 0x403450: cldb_down (cldb.c:414)
==4506==    by 0x409A58: main (server.c:1143)
==4506==  Address 0x4f005e3 is 4,083 bytes inside a block of size 4,096 alloc'd
==4506==    at 0x4A0515D: malloc (vg_replace_malloc.c:195)
==4506==    by 0x31BE135237: __os_malloc (os_alloc.c:253)
==4506==    by 0x31BE1257FC: __memp_pgwrite (mp_bh.c:385)
==4506==    by 0x31BE125954: __memp_bhwrite (mp_bh.c:168)
==4506==    by 0x31BE134482: __memp_sync_int (mp_sync.c:530)
==4506==    by 0x31BE0CBD66: __db_sync (db_am.c:706)
==4506==    by 0x31BE0CA652: __db_refresh (db.c:820)
==4506==    by 0x31BE0CA8C0: __db_close (db.c:695)
==4506==    by 0x31BE0E64A7: __db_close_pp (db_iface.c:253)
==4506==    by 0x403450: cldb_down (cldb.c:414)
==4506==    by 0x409A58: main (server.c:1143)

==4506== Uninitialised value was created by a stack allocation
==4506== at 0x404878: cldb_lock_add (cldb.c:967)
==4506==
==4506==
==4506== 1 errors in context 2 of 7:
==4506== Syscall param pwrite64(buf) points to uninitialised byte(s)

==4506==    at 0x31BB40EDE3: __pwrite_nocancel (syscall-template.S:82)
==4506==    by 0x31BE137AD0: __os_io (os_rw.c:92)
==4506==    by 0x31BE1207C4: __log_write (log_put.c:1227)
==4506==    by 0x31BE121477: __log_flush_int (log_put.c:1010)
==4506==    by 0x31BE122CDC: __log_put (log_put.c:496)
==4506==    by 0x31BE146077: __txn_regop_log (txn_auto.c:234)
==4506==    by 0x31BE142A32: __txn_commit (txn.c:656)
==4506==    by 0x31BE142ABE: __txn_commit_pp (txn.c:517)
==4506==    by 0x4073A1: msg_lock (msg.c:1150)
==4506==    by 0x407869: udp_rx_handle (server.c:164)
==4506==    by 0x407BF3: udp_rx (server.c:265)
==4506==    by 0x408A3A: udp_srv_event (server.c:640)
==4506==    by 0x409676: main_loop (server.c:1026)
==4506==    by 0x409A0E: main (server.c:1135)

==4506== Address 0x52d5380 is not stack'd, malloc'd or (recently) free'd ==4506== Uninitialised value was created by a stack allocation
==4506== at 0x404878: cldb_lock_add (cldb.c:967)
==4506==
==4506==
==4506== 1 errors in context 3 of 7:
==4506== Conditional jump or move depends on uninitialised value(s)

==4506==    at 0x31BE121A78: __log_putr (log_put.c:732)
==4506==    by 0x31BE1228B8: __log_put (log_put.c:464)
==4506==    by 0x31BE064E28: __ham_insdel_log (hash_auto.c:250)
==4506==    by 0x31BE06E86F: __ham_add_el (hash_page.c:2197)
==4506==    by 0x31BE05E5EC: __hamc_put (hash.c:1089)
==4506==    by 0x31BE0D97E5: __dbc_iput (db_cam.c:2103)
==4506==    by 0x31BE0DBE0C: __dbc_put (db_cam.c:2016)
==4506==    by 0x31BE0CCEA5: __db_put (db_am.c:477)
==4506==    by 0x31BE0E3DE2: __db_put_pp (db_iface.c:1579)
==4506==    by 0x404A52: cldb_lock_add (cldb.c:1016)
==4506==    by 0x40735E: msg_lock (msg.c:1140)
==4506==    by 0x407869: udp_rx_handle (server.c:164)
==4506==    by 0x407BF3: udp_rx (server.c:265)
==4506==    by 0x408A3A: udp_srv_event (server.c:640)
==4506==    by 0x409676: main_loop (server.c:1026)
==4506==    by 0x409A0E: main (server.c:1135)

==4506== Uninitialised value was created by a stack allocation
==4506== at 0x404878: cldb_lock_add (cldb.c:967)

Then, just for grins, I ran chunkd (via make check) under valgrind with --gen-suppressions=all, which generated 27000 lines of suppressions, and extracted the unique ones. Even those could be further consolidated with frame-level "..." wildcards (see http://valgrind.org/docs/manual/manual-core.html)
but that would take careful manual work.

Here's the result. Obviously, before using these, you'll want to ensure that each really does represent an ignorable condition. For the ssl-related signatures, it's pretty easy to assume. For most of the others, it'll require careful work.

{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
fun:g_malloc
fun:g_strndup
fun:cfg_elm_text
fun:g_markup_parse_context_parse
fun:read_config
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
fun:cfg_elm_end_listen
fun:cfg_elm_end
fun:g_markup_parse_context_parse
fun:read_config
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:memalign
fun:posix_memalign
fun:slab_allocator_alloc_chunk
fun:g_slice_alloc
fun:g_slist_prepend
fun:g_strsplit
fun:g_get_language_names
fun:g_thread_init_glib
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:memalign
fun:posix_memalign
fun:slab_allocator_alloc_chunk
fun:g_slice_alloc
fun:g_array_sized_new
fun:g_static_private_set
fun:g_get_filename_charsets
fun:_g_convert_thread_init
fun:g_thread_init_glib
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_bin2bn
fun:bnrand
fun:bn_rand_range
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_cmp
fun:bn_rand_range
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_ucmp
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Value8
fun:bn_mul_mont
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
}
{
<insert_a_suppression_name_here>
Memcheck:Value8
fun:bn_mul_mont
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
obj:*
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:calloc
fun:cli_writeq
fun:cli_err
fun:login_user
fun:cli_evt_exec_req
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
obj:/usr/lib64/libtokyocabinet.so.9.6.0
fun:tchdbget
fun:fs_table_open
fun:volume_open
fun:cli_evt_exec_req
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
fun:object_del
fun:cli_evt_exec_req
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:calloc
fun:_dl_allocate_tls
fun:pthread_create@@GLIBC_2.2.5
fun:g_thread_create_posix_impl
fun:g_thread_create_full
fun:ncld_sess_open
fun:cldu_set_cldc
fun:cld_begin
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:calloc
fun:_dl_allocate_tls
fun:pthread_create@@GLIBC_2.2.5
fun:g_thread_create_posix_impl
fun:g_thread_create_full
fun:g_thread_pool_start_thread
fun:g_thread_pool_push
fun:object_cp
fun:cli_evt_exec_req
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:calloc
fun:cli_wr_sendfile
fun:object_read_bytes
fun:object_get
fun:cli_evt_exec_req
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
fun:xdr_string
fun:xdr_cld_pkt_hdr
fun:cldc_receive_pkt
fun:cldc_udp_receive_pkt
fun:ncld_sess_thr
fun:g_thread_create_proxy
fun:start_thread
fun:clone
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:malloc
fun:realloc
fun:g_realloc
fun:g_array_maybe_expand
fun:g_array_sized_new
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:memalign
fun:posix_memalign
fun:slab_allocator_alloc_chunk
fun:g_slice_alloc
fun:g_list_append
fun:fs_list_objs
fun:volume_list
fun:cli_evt_exec_req
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:memalign
fun:posix_memalign
fun:slab_allocator_alloc_chunk
fun:g_slice_alloc
fun:g_string_sized_new
fun:g_markup_escape_text
fun:volume_list
fun:cli_evt_exec_req
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Leak
fun:calloc
fun:cli_writeq
fun:cli_write_list
fun:cli_resp_xml
fun:volume_list
fun:cli_evt_exec_req
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:MOD_EXP_CTIME_COPY_TO_PREBUF
fun:BN_mod_exp_mont_consttime
fun:RSA_eay_mod_exp
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_mod_mul_montgomery
fun:BN_mod_exp_mont_consttime
fun:RSA_eay_mod_exp
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_mod_exp_mont_consttime
fun:RSA_eay_mod_exp
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_ucmp
fun:BN_mod_exp_mont_consttime
fun:RSA_eay_mod_exp
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_div
fun:RSA_eay_mod_exp
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_div
fun:BN_nnmod
fun:BN_mod_mul
fun:BN_BLINDING_convert_ex
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_mul
fun:BN_mod_mul
fun:BN_BLINDING_convert_ex
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:bn_mul_recursive
fun:BN_mul
fun:BN_mod_mul
fun:BN_BLINDING_convert_ex
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:bn_cmp_words
fun:bn_mul_recursive
fun:BN_mul
fun:BN_mod_mul
fun:BN_BLINDING_convert_ex
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_from_montgomery_word
fun:BN_from_montgomery
fun:BN_mod_exp_mont
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Value8
fun:BN_from_montgomery_word
fun:BN_from_montgomery
fun:BN_mod_exp_mont
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_mod_mul_montgomery
fun:BN_mod_exp_mont
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_mod_exp_mont
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_ucmp
fun:BN_mod_exp_mont
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_div
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_num_bits_word
fun:BN_num_bits
fun:BN_div
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Value8
fun:bn_add_words
fun:bn_mul_recursive
fun:BN_mul
fun:BN_mod_mul
fun:BN_BLINDING_convert_ex
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Value8
fun:bn_sub_words
fun:bn_mul_recursive
fun:BN_mul
fun:BN_mod_mul
fun:BN_BLINDING_convert_ex
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Value8
fun:BN_num_bits_word
fun:BN_num_bits
fun:BN_div
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_lshift
fun:BN_div
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_uadd
fun:BN_add
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_rshift
fun:BN_div
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:BN_mul
fun:BN_mod_inverse
fun:BN_BLINDING_create_param
fun:RSA_setup_blinding
fun:rsa_get_blinding
fun:RSA_eay_private_decrypt
fun:ssl3_get_client_key_exchange
fun:ssl3_accept
fun:cli_evt_ssl_accept
fun:tcp_cli_event
fun:main_loop
fun:main
}

applied