fs: dlm: locking and memory fixes
Alexander Aring
maintainercover fs: dlm: locking and memory fixes
Hi,
this patch series contains some locking and memory issues which I found while working on a bigger fix to make dlm secured against tcpkill.
- Alex
Alexander Aring (7):
fs: dlm: synchronize dlm before shutdown
fs: dlm: make connection hash lockless
fs: dlm: fix dlm_local_addr memory leak
fs: dlm: fix configfs memory leak
fs: dlm: move free writequeue into con free
fs: dlm: handle possible othercon writequeues
fs: dlm: use free_con to free connection
fs/dlm/Kconfig | 1 + fs/dlm/config.c | 3 ++ fs/dlm/lowcomms.c | 122 +++++++++++++++++++++------------------------- 3 files changed, 60 insertions(+), 66 deletions(-)
Signature · 1 line
2.26.2
Alexander Aring
maintainerPATCH 1/7 fs: dlm: synchronize dlm before shutdown
This patch moves the dlm workqueue dlm synchronization before shutdown handling. The patch just flushes all pending work before starting to shutdown the connection. At least for the send_workqeue we should flush the workqueue to make sure there is no new connection handling going on as dlm_allow_conn switch is turned to false before.
--- fs/dlm/lowcomms.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
fs/dlm/lowcomms.c +6 -4
@@ -1624,10 +1624,6 @@ static void work_flush(void)struct hlist_node *n;struct connection *con; -if (recv_workqueue) -flush_workqueue (recv_workqueue); -if (send_workqueue) -flush_workqueue (send_workqueue); do { ok = 1; foreach_conn(stop_conn);
@@ -1659,6 +1655,12 @@ void dlm_lowcomms_stop(void)mutex_lock (&connections_lock); dlm_allow_conn =0 ;mutex_unlock (&connections_lock); + +if (recv_workqueue) +flush_workqueue (recv_workqueue); +if (send_workqueue) +flush_workqueue (send_workqueue); +foreach_conn (shutdown_conn);work_flush ();clean_writequeues ();
Signature · 1 line
2.26.2
Alexander Aring
maintainerPATCH 2/7 fs: dlm: make connection hash lockless
In reply to Alexander AringThere are some problems with the connections_lock. During my experiements I saw sometimes circular dependencies with sock_lock. The reason here might be code parts which runs nodeid2con() before or after sock_lock is acquired.
Another issue are missing locks in for_conn() iteration. Maybe this works fine because for_conn() is running in a context where connection_hash cannot be manipulated by others anymore.
However this patch changes the connection_hash to be protected by sleepable rcu. The hotpath function __find_con() is implemented lockless as it is only a reader of connection_hash and this hopefully fixes the circular locking dependencies. The iteration for_conn() will still call some sleepable functionality, that's why we use sleepable rcu in this case.
This patch removes the kmemcache functionality as I think I need to make some free() functionality via call_rcu(). However allocation time isn't here an issue. The dlm_allow_con will not be protected by a lock anymore as I think it's enough to just set and flush workqueues afterwards.
--- fs/dlm/Kconfig | 1 + fs/dlm/lowcomms.c | 86 ++++++++++++++++++++--------------------------- 2 files changed, 37 insertions(+), 50 deletions(-)
fs/dlm/Kconfig +1 -0
@@ -4,6 +4,7 @@ menuconfig DLM depends on INET depends on SYSFS && CONFIGFS_FS && (IPV6 || IPV6=n) select IP_SCTP + select SRCU help A general purpose distributed lock manager for kernel or userspace applications.
fs/dlm/lowcomms.c +36 -50
209 lines not on this page · raw message
Signature · 1 line
2.26.2
Alexander Aring
maintainerPATCH 3/7 fs: dlm: fix dlm_local_addr memory leak
In reply to Alexander AringThis patch fixes the following memory detected by kmemleak and umount gfs2 filesystem which removed the last lockspace:
unreferenced object 0xffff9264f4f48f00 (size 128):
comm "mount", pid 425, jiffies 4294690253 (age 48.159s) hex dump (first 32 bytes):
02 00 52 48 c0 a8 7a fb 00 00 00 00 00 00 00 00 ..RH..z.........
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<0000000067a34940>] kmemdup+0x18/0x40
[<00000000c935f9ab>] init_local+0x4c/0xa0
[<00000000bbd286ef>] dlm_lowcomms_start+0x28/0x160
[<00000000a86625cb>] dlm_new_lockspace+0x7e/0xb80
[<000000008df6cd63>] gdlm_mount+0x1cc/0x5de
[<00000000b67df8c7>] gfs2_lm_mount.constprop.0+0x1a3/0x1d3
[<000000006642ac5e>] gfs2_fill_super+0x717/0xba9
[<00000000d3ab7118>] get_tree_bdev+0x17f/0x280
[<000000001975926e>] gfs2_get_tree+0x21/0x90
[<00000000561ce1c4>] vfs_get_tree+0x28/0xc0
[<000000007fecaf63>] path_mount+0x434/0xc00
[<00000000636b9594>] __x64_sys_mount+0xe3/0x120
[<00000000cc478a33>] do_syscall_64+0x33/0x40
[<00000000ce9ccf01>] entry_SYSCALL_64_after_hwframe+0x44/0xa9--- fs/dlm/lowcomms.c | 9 +++++++++ 1 file changed, 9 insertions(+)
fs/dlm/lowcomms.c +9 -0
@@ -1234,6 +1234,14 @@ static void init_local(void) } } +static void deinit_local (void ) +{ +int i; + +for (i =0 ; i < dlm_local_count; i++) +kfree (dlm_local_addr[i]); +} +/* Initialise SCTP socket and bind to all interfaces */ static int sctp_listen_for_all (void ) {
@@ -1663,6 +1671,7 @@ void dlm_lowcomms_stop(void)clean_writequeues ();foreach_conn (free_conn);work_stop (); +deinit_local (); }int dlm_lowcomms_start (void )
Signature · 1 line
2.26.2
Alexander Aring
maintainerPATCH 4/7 fs: dlm: fix configfs memory leak
In reply to Alexander AringThis patch fixes the following memory detected by kmemleak and umount gfs2 filesystem which removed the last lockspace:
unreferenced object 0xffff9264f482f600 (size 192):
comm "dlm_controld", pid 325, jiffies 4294690276 (age 48.136s) hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 6e 6f 64 65 73 00 00 00 ........nodes...
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000060481d7>] make_space+0x41/0x130
[<000000008d905d46>] configfs_mkdir+0x1a2/0x5f0
[<00000000729502cf>] vfs_mkdir+0x155/0x210
[<000000000369bcf1>] do_mkdirat+0x6d/0x110
[<00000000cc478a33>] do_syscall_64+0x33/0x40
[<00000000ce9ccf01>] entry_SYSCALL_64_after_hwframe+0x44/0xa9The patch just remembers the "nodes" entry pointer in space as I think it's created as subdirectory when parent "spaces" is created. In function drop_space() we will lost the pointer reference to nds because configfs_remove_default_groups(). However as this subdirectory is always available when "spaces" exists it will just be freed when "spaces" will be freed.
--- fs/dlm/config.c | 3 +++ 1 file changed, 3 insertions(+)
fs/dlm/config.c +3 -0
@@ -221,6 +221,7 @@ struct dlm_space {struct list_head members;struct mutex members_lock;int members_count; +struct dlm_nodes *nds; };struct dlm_comms {
@@ -430,6 +431,7 @@ static struct config_group *make_space(struct config_group *g, const char *name)INIT_LIST_HEAD (&sp->members);mutex_init (&sp->members_lock); sp->members_count =0 ; + sp->nds = nds;return &sp->group; fail:
@@ -451,6 +453,7 @@ static void drop_space(struct config_group *g, struct config_item *i)static void release_space (struct config_item *i) {struct dlm_space *sp =config_item_to_space (i); +kfree (sp->nds);kfree (sp); }
Signature · 1 line
2.26.2
Alexander Aring
maintainerPATCH 5/7 fs: dlm: move free writequeue into con free
In reply to Alexander AringThis patch just move the free of struct connection member writequeue into the functionality when struct connection will be freed instead of doing two iterations.
--- fs/dlm/lowcomms.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-)
fs/dlm/lowcomms.c +1 -8
@@ -1550,13 +1550,6 @@ static void process_send_sockets(struct work_struct *work)send_to_sock (con); } - -/* Discard all entries on the write queues */ -static void clean_writequeues (void ) -{ -foreach_conn (clean_one_writequeue); -} -static void work_stop (void ) {if (recv_workqueue)
@@ -1620,6 +1613,7 @@ static void free_conn(struct connection *con)spin_lock (&connections_lock);hlist_del_rcu (&con->list);spin_unlock (&connections_lock); +clean_one_writequeue (con);kfree_rcu (con, rcu); }
@@ -1668,7 +1662,6 @@ void dlm_lowcomms_stop(void)foreach_conn (shutdown_conn);work_flush (); -clean_writequeues ();foreach_conn (free_conn);work_stop ();deinit_local ();
Signature · 1 line
2.26.2
Alexander Aring
maintainerPATCH 6/7 fs: dlm: handle possible othercon writequeues
In reply to Alexander AringThis patch adds free of possible other writequeue entries in othercon member of struct connection.
--- fs/dlm/lowcomms.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/dlm/lowcomms.c +4 -2
@@ -1608,11 +1608,13 @@ static void shutdown_conn(struct connection *con)static void free_conn (struct connection *con) {close_connection (con, true, true, true); -if (con->othercon) -kfree_rcu (con->othercon, rcu);spin_lock (&connections_lock);hlist_del_rcu (&con->list);spin_unlock (&connections_lock); +if (con->othercon) { +clean_one_writequeue (con->othercon); +kfree_rcu (con->othercon, rcu); + }clean_one_writequeue (con);kfree_rcu (con, rcu); }
Signature · 1 line
2.26.2
Alexander Aring
maintainerPATCH 7/7 fs: dlm: use free_con to free connection
In reply to Alexander AringThis patch use free_con() functionality to free the listen connection if listen fails. It also fixes an issue that a freed resource is still part of the connection_hash as hlist_del() is not called in this case. The only difference is that free_con() handles othercon as well, but this is never been set for the listen connection.
--- fs/dlm/lowcomms.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
fs/dlm/lowcomms.c +2 -4
@@ -1704,10 +1704,8 @@ int dlm_lowcomms_start(void) fail_unlisten: dlm_allow_conn =0 ; con =nodeid2con (0 ,0 ); -if (con) { -close_connection (con, false, true, true); -kfree_rcu (con, rcu); - } +if (con) +free_conn (con); fail:return error; }
Signature · 1 line
2.26.2