add multichannel support
Subsystems (2)
v4 cover add multichannel support
Hello,
I have very little time to work on this during the work week due to annoying^W lovely customers, company stuff and backporting work but during this long weekend of festivities I somehow mustered the courage to dig in again...
So without further ado, here is another iteration of the multichannel patchset.
Couple of changes since last time:
* do round-robin on the channels (cycle over them on requests)
* reuse master client guid for new channel (fixes binding against samba)
* simplification of crypto key lookup and potential race condition fix
* properly discard channels on session closing
* better display of channels info in DebugData
Usage
=====
Make sure your server exposes multiple interfaces and mount with:
-o multichannel,max_channels=2
To have a total of 2 connections. One master and one extra channel.
* Each call to ...send_recv() will alternate between the 2 channels.
* You can check /proc/fs/cifs/DebugData for channel infos
Overview
========
Multichannel is part of SMB3 and above. It lets the client "share" the same SMB session over multiple TCP (or RDMA) connections.
* First, a regular connection is made.
client server
| | | ----------- negprot req (unsigned, guid=0xABC)---------> | | | | <---------- negprot res (unsigned) ------------------- | | | | ---------- sesssetup req (sesid=0x000, unsigned) ------> | | | | <--------- sesssetup res (sesid=0x123, signed) --------- | | | | (client now has signing/encryption/decryption keys) | | | | | | ----------- ... (signed) -------------------------------> | | <------------------------------------------------------- | | |
* The client then queries the network interface available on the server via an FSCTL (this already implemented in cifs.ko)
* For each new channel, the client opens another connection, but reuses the client guid, and at session setup it sets a binding flag to say "this is a new channel, please make this session an alias to sess id 0x123"
* Since the session is an alias, you can use TreeCons, files, etc interchangeably between channels as if it was the same transport.
* Channels use the same encryption/decryption keys from the session BUT each channel has its own signing key.
* When you see "signed" in lowercase, it's signed using the session/master key. When you see "SIGNED" in uppercase, it's signed with the CHANNEL key.
Here is the traffic for opening and using new channels:
| | --------- negprot req (unsigned, guid=0xABC) ----------> | o | | o | <-------- negprog res (unsigned) ---------------------- | o BINDING | | o PART | --------- sesssetup req (sesid=0x123, binding, signed)-> | o | | o | <-------- sesssetup res (sesid=0x123, SIGNED) --------- | o | | | (from then on, client needs to use CHANNEL key | | for signing) | | | | --------- ... (SIGNED) --------------------------------> | | <-------------------------------------------------------- | | |
This signing key business is tricky to get right. As usual refer to MS-SMB2 for more boring details.
cifs.ko implementation
======================
Data structure
--------------
We add an array of struct cifs_chan to cifs_ses. A channel has a server pointer and a signing key.
struct cifs_ses {
struct TCP_Server_Info *server; <-- master con
u8 smb3signingkey[...]; <-- master sign key
u8 smb3encryptionkey[...]; <-- extra channels & master
u8 smb3decryptionkey[...]; <-- extra channels & master
bool binding; <--- true during BINDING PART
...
struct cifs_chan {
struct TCP_Server_Info *server; <-- channel con u8 signkey[SMB3_SIGN_KEY_SIZE]; <-- channel sign key
} chans[16];
};
New channel connections (TCP_Server_Info) end up in the usual global cifs_tcp_ses_list linked list but also in the ses channel array.
,-------------> global cifs_tcp_ses_list <-------------------------.
| |
'- TCP_Server_Info <--> TCP_Server_Info <--> TCP_Server_Info <-'
(master con) (chan#1 con) (chan#2 con)
| ^ ^ ^
v '--------------------|--------------------'
cifs_ses |
- chan_count = 3 |- chans[].server --------------'
Some important details:
* ses->server still points to the master con.
* ses->server == chans[0].server
* chan#1 and chan#2 have an empty server->smb_ses_list
Channel/session establishment
-----------------------------
* When opening channels we check ses->binding to know that we are in the BINDING PART. In which case we don't create a new session object but merely reuse the existing one.
* chan_count only keeps track of fully established channels. During the BINDING PART there is actually chan_count+1 channels, the last one being the channel in the process of being created.
NOTE: we could add a state field to cifs_chan and make cifs_count
always accurate. Not sure if it's better.
Channel dispatch
----------------
On syscalls, as we get lower and lower in the bowels of cifs.ko we eventually reach the transport layer. There we often pass a ses pointer to code that assumes the transport can be reached via ses->server.
* A big change was to decouple ses from server in those codepaths. So that we can select a channel and make the code use that one instead of the master.
* That often means passing ses AND server and replacing ses->server by server.
Signing and encryption
----------------------
* When generating the keys we make sure to not overwrite enc/dec keys for new channels and store the sign key in cifs_chan.
* When sending and receiving a message we have to search for the right keys by searching the whole cifs_tcp_ses_list instead of just looking at the master con sessions.
Aurelien Aptel (6):
cifs: sort interface list by speed
cifs: add multichannel mount options and data structs
cifs: add server param
cifs: switch servers depending on binding state
cifs: try opening channels after mounting
cifs: dump channel info in DebugData
fs/cifs/cifs_debug.c | 35 +++++++- fs/cifs/cifs_spnego.c | 2 +- fs/cifs/cifsfs.c | 4 + fs/cifs/cifsglob.h | 45 +++++++++- fs/cifs/cifsproto.h | 8 ++ fs/cifs/connect.c | 92 ++++++++++++++++---- fs/cifs/sess.c | 218 +++++++++++++++++++++++++++++++++++++++++++++++- fs/cifs/smb2misc.c | 37 +++++--- fs/cifs/smb2ops.c | 31 +++++-- fs/cifs/smb2pdu.c | 109 ++++++++++++++---------- fs/cifs/smb2proto.h | 3 +- fs/cifs/smb2transport.c | 165 +++++++++++++++++++++++++++--------- fs/cifs/transport.c | 18 +++- 13 files changed, 636 insertions(+), 131 deletions(-)
--
2.16.4
PATCH v4 1/6 cifs: sort interface list by speed
New channels are going to be opened by walking the list sequentially, so by sorting it we will connect to the fastest interfaces first.
--- fs/cifs/smb2ops.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
fs/cifs/smb2ops.c +11 -0
@@ -10,6 +10,7 @@#include <linux/falloc.h> #include <linux/scatterlist.h> #include <linux/uuid.h> +#include <linux/sort.h> #include <crypto/aead.h> #include "cifsglob.h" #include "smb2pdu.h"
@@ -558,6 +559,13 @@ parse_server_interfaces(struct network_interface_info_ioctl_rsp *buf,return rc; } +static int compare_iface (const void *ia,const void *ib) +{ +const struct cifs_server_iface *a = (struct cifs_server_iface *)ia; +const struct cifs_server_iface *b = (struct cifs_server_iface *)ib; + +return a->speed == b->speed ?0 : (a->speed > b->speed ?-1 :1 ); +}static int SMB3_request_interfaces (const unsigned int xid,struct cifs_tcon *tcon)
@@ -587,6 +595,9 @@ SMB3_request_interfaces(const unsigned int xid, struct cifs_tcon *tcon)if (rc) goto out; +/* sort interfaces from fastest to slowest */ +sort (iface_list, iface_count,sizeof (*iface_list), compare_iface,NULL ); +spin_lock (&ses->iface_lock);kfree (ses->iface_list); ses->iface_list = iface_list;
Signature · 1 line
2.16.4
Re: [PATCH v4 1/6] cifs: sort interface list by speed
сб, 2 нояб. 2019 г. в 18:21, Aurelien Aptel <[email protected]>:
Quoted 46 lines
New channels are going to be opened by walking the list sequentially, so by sorting it we will connect to the fastest interfaces first.
Signed-off-by Aurelien Aptel <[email protected]>--- fs/cifs/smb2ops.c | 11 +++++++++++ 1 file changed, 11 insertions(+)fs/cifs/smb2ops.c +11
-1@@ -10,6 +10,7 @@#include <linux/falloc.h> #include <linux/scatterlist.h> #include <linux/uuid.h> +#include <linux/sort.h> #include <crypto/aead.h> #include "cifsglob.h" #include "smb2pdu.h" @@ -558,6 +559,13 @@ parse_server_interfaces(struct network_interface_info_ioctl_rsp *buf,return rc; } +static int compare_iface (const void *ia,const void *ib) +{ +const struct cifs_server_iface *a = (struct cifs_server_iface *)ia; +const struct cifs_server_iface *b = (struct cifs_server_iface *)ib; + +return a->speed == b->speed ?0 : (a->speed > b->speed ?-1 :1 ); +}static int SMB3_request_interfaces (const unsigned int xid,struct cifs_tcon *tcon)@@ -587,6 +595,9 @@ SMB3_request_interfaces(const unsigned int xid, struct cifs_tcon *tcon)if (rc) goto out; +/* sort interfaces from fastest to slowest */ +sort (iface_list, iface_count,sizeof (*iface_list), compare_iface,NULL ); +spin_lock (&ses->iface_lock);kfree (ses->iface_list); ses->iface_list = iface_list; --
Quoted 2 lines
2.16.4
Looks good at the first glance, thanks!
@Steve, you may add
to this and other patches in the series.
--
Best regards,
Pavel Shilovsky
RE: [EXTERNAL] Re: [PATCH v4 1/6] cifs: sort interface list by speed
Quoted 9 lines
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Pavel Shilovsky
Sent: Monday, November 25, 2019 4:29 PM
To: Aurelien Aptel <[email protected]>
Cc: linux-cifs <[email protected]>; Steve French <[email protected]>
Subject: [EXTERNAL] Re: [PATCH v4 1/6] cifs: sort interface list by speed
Quoted 4 lines
сб, 2 нояб. 2019 г. в 18:21, Aurelien Aptel <[email protected]>:
Quoted 3 lines
New channels are going to be opened by walking the list sequentially, so by sorting it we will connect to the fastest interfaces first.
Sorting by speed is definitely appropriate, but sorting the other multichannel attributes is just as important. For example, if the RDMA attribute is set, the address should actually be excluded for non-RDMA connections (a second, non-RDMA entry is included, if the interface supports both). And, the RSS attribute indicates a "better" destination than non-RSS for a given speed, because it is more efficient at local traffic management.
Suggest a followon effort to take advantage of these, by excluding ineligible paths, and raising the sort priority of others.
Tom.
Quoted 58 lines
fs/cifs/smb2ops.c parse_server_interfaces(struct
Quoted 18 lines
Signed-off-by Aurelien Aptel <[email protected]>--- fs/cifs/smb2ops.c | 11 +++++++++++ 1 file changed, 11 insertions(+)fs/cifs/smb2ops.c +1
-0@@ -10,6 +10,7 @@#include <linux/falloc.h> #include <linux/scatterlist.h> #include <linux/uuid.h> +#include <linux/sort.h> #include <crypto/aead.h> #include "cifsglob.h" #include "smb2pdu.h" @@ -558,6 +559,13 @@ parse_server_interfaces(struct
network_interface_info_ioctl_rsp*buf,fs/cifs/smb2ops.c SMB3_request_interfaces(const unsigned int xid,
Quoted 14 lines
fs/cifs/smb2ops.c +7
-0return rc; } +static int compare_iface (const void *ia,const void *ib) +{ +const struct cifs_server_iface *a = (struct cifs_server_iface *)ia; +const struct cifs_server_iface *b = (struct cifs_server_iface *)ib; + +return a->speed == b->speed ?0 : (a->speed > b->speed ?-1 :1 ); +}static int SMB3_request_interfaces (const unsigned int xid,struct cifs_tcon *tcon)@@ -587,6 +595,9 @@ SMB3_request_interfaces(const unsigned int xid,struct
cifs_tcon*tcon)Quoted 10 lines
fs/cifs/smb2ops.c +3
-1if (rc) goto out; +/* sort interfaces from fastest to slowest */ +sort (iface_list, iface_count,sizeof (*iface_list), compare_iface,NULL ); +spin_lock (&ses->iface_lock);kfree (ses->iface_list); ses->iface_list = iface_list; --Quoted 2 lines
2.16.4
Looks good at the first glance, thanks!
@Steve, you may add
Acked-by Pavel Shilovsky <[email protected]>to this and other patches in the series.
--
Best regards,
Pavel Shilovsky
RE: [EXTERNAL] Re: [PATCH v4 1/6] cifs: sort interface list by speed
Hi,
Quoted 7 lines · Tom Talpey
Sorting by speed is definitely appropriate, but sorting the other multichannel attributes is just as important. For example, if the RDMA attribute is set, the address should actually be excluded for non-RDMA connections (a second, non-RDMA entry is included, if the interface supports both). And, the RSS attribute indicates a "better" destination than non-RSS for a given speed, because it is more efficient at local traffic management.
Note that the way the list is used has been altered in a later patch here:
https://lore.kernel.org/linux-cifs/[email protected]/T/#u
Cheers,
Signature · 4 lines
Aurélien Aptel / SUSE Labs Samba Team GPG: 1839 CB5F 9F5B FB9B AA97 8C99 03C8 A49B 521B D5D3 SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)
RE: [EXTERNAL] Re: [PATCH v4 1/6] cifs: sort interface list by speed
Quoted 20 lines
-----Original Message-----
From: Aurélien Aptel <[email protected]>
Sent: Tuesday, November 26, 2019 11:54 AM
To: Tom Talpey <[email protected]>; Pavel Shilovsky <[email protected]>
Cc: linux-cifs <[email protected]>; Steve French <[email protected]>
Subject: RE: [EXTERNAL] Re: [PATCH v4 1/6] cifs: sort interface list by speedHi,
Quoted 7 lines · Tom Talpey
Sorting by speed is definitely appropriate, but sorting the other multichannel attributes is just as important. For example, if the RDMA attribute is set, the address should actually be excluded for non-RDMA connections (a second, non-RDMA entry is included, if the interface supports both). And, the RSS attribute indicates a "better" destination than non-RSS for a given speed, because it is more efficient at local traffic management.
Quoted 1 line
Note that the way the list is used has been altered in a later patch
Hmm, that patch does help by taking RSS into account, but it does not take RDMA into consideration. The logic for that may be more complex, as it would need to determine whether an RDMA mount was desired, and an adapter available. And it looks like the code will try the RDMA addresses from other transport protocols.
I also think it is more than a little too "heroic". Retrying seems unnecessary, doesn't the code refresh the list from time to time and reattempt connections? That's what the Windows client does, anyway. I would suggest avoiding any kind of retry except on the very first connection, and even then, many admins will want a fast-fail. Three TCP retries can take a long time!
Tom.
Quoted 15 lines
here:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.ker
nel.org%2Flinux-cifs%2F20191120161559.30295-2- aaptel%40suse.com%2FT%2F%23u&data=02%7C01%7Cttalpey%40micros oft.com%7Cf76a99bd50ea4fbf4f7908d772914946%7C72f988bf86f141af91ab2 d7cd011db47%7C1%7C0%7C637103840632982128&sdata=OJxqwDoV8C CYNo69cgR9nwmc1xGR22s8J9Y9GDiV74M%3D&reserved=0Cheers,
--
Aurélien Aptel / SUSE Labs Samba Team
GPG: 1839 CB5F 9F5B FB9B AA97 8C99 03C8 A49B 521B D5D3
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nürnberg, DE
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 247165 (AG München)
PATCH v4 2/6 cifs: add multichannel mount options and data structs
In reply to Aurelien Apteladds:
- [no]multichannel to enable/disable multichannel
- max_channels=N to control how many channels to create
these options are then stored in the volume struct.
- store channels and max_channels in cifs_ses
--- fs/cifs/cifsfs.c | 4 ++++ fs/cifs/cifsglob.h | 16 ++++++++++++++++ fs/cifs/connect.c | 38 ++++++++++++++++++++++++++++++++++++-- 3 files changed, 56 insertions(+), 2 deletions(-)
fs/cifs/cifsfs.c +4 -0
@@ -613,6 +613,10 @@ cifs_show_options(struct seq_file *s, struct dentry *root)/* convert actimeo and display it in seconds */ seq_printf (s,",actimeo=%lu" , cifs_sb->actimeo /HZ ); +if (tcon->ses->chan_max >1 ) +seq_printf (s,",multichannel,max_channel=%zu" , + tcon->ses->chan_max); +return 0 ; }
fs/cifs/cifsglob.h +16 -0
@@ -591,6 +591,10 @@ struct smb_vol { bool resilient:1; /* noresilient not required since not fored for CA */ bool domainauto:1; bool rdma:1; + bool multichannel:1; + bool use_client_guid:1; +/* reuse existing guid for multichannel */ +u8 client_guid[SMB2_CLIENT_GUID_SIZE ];unsigned int bsize;unsigned int rsize;unsigned int wsize;
@@ -607,6 +611,7 @@ struct smb_vol {__u64 snapshot_time;/* needed for timewarp tokens */ __u32 handle_timeout;/* persistent and durable handle timeout in ms */ unsigned int max_credits;/* smb3 max_credits 10 < credits < 60000 */ +unsigned int max_channels;__u16 compression;/* compression algorithm 0xFFFF default 0=disabled */ bool rootfs:1; /* if it's a SMB root file system */ };
@@ -953,6 +958,11 @@ struct cifs_server_iface {struct sockaddr_storage sockaddr; }; +struct cifs_chan { +struct TCP_Server_Info *server; +__u8 signkey[SMB3_SIGN_KEY_SIZE ]; +}; +/* * Session structure. One of these for each uid session with a particular host */
@@ -1002,6 +1012,12 @@ struct cifs_ses {struct cifs_server_iface *iface_list;size_t iface_count;unsigned long iface_last_update;/* jiffies */ + +#define CIFS_MAX_CHANNELS 16 +struct cifs_chan chans[CIFS_MAX_CHANNELS ]; +size_t chan_count; +size_t chan_max; +atomic_t chan_seq;/* round robin state */ }; static inline bool
fs/cifs/connect.c +36 -2
@@ -97,6 +97,7 @@ enum { Opt_persistent, Opt_nopersistent, Opt_resilient, Opt_noresilient, Opt_domainauto, Opt_rdma, Opt_modesid, Opt_rootfs, + Opt_multichannel, Opt_nomultichannel, Opt_compress, /* Mount options which take numeric value */
@@ -106,7 +107,7 @@ enum { Opt_min_enc_offload, Opt_blocksize, Opt_rsize, Opt_wsize, Opt_actimeo, Opt_echo_interval, Opt_max_credits, Opt_handletimeout, - Opt_snapshot, + Opt_snapshot, Opt_max_channels,/* Mount options which take string value */ Opt_user, Opt_pass, Opt_ip,
@@ -199,6 +200,8 @@ static const match_table_t cifs_mount_option_tokens = { { Opt_noresilient, "noresilienthandles"}, { Opt_domainauto, "domainauto"}, { Opt_rdma, "rdma"}, + { Opt_multichannel, "multichannel" }, + { Opt_nomultichannel, "nomultichannel" }, { Opt_backupuid, "backupuid=%s" }, { Opt_backupgid, "backupgid=%s" },
@@ -218,6 +221,7 @@ static const match_table_t cifs_mount_option_tokens = { { Opt_echo_interval, "echo_interval=%s" }, { Opt_max_credits, "max_credits=%s" }, { Opt_snapshot, "snapshot=%s" }, + { Opt_max_channels, "max_channels=%s" }, { Opt_compress, "compress=%s" }, { Opt_blank_user, "user=" },
@@ -1672,6 +1676,10 @@ cifs_parse_mount_options(const char *mountdata, const char *devname, vol->echo_interval =SMB_ECHO_INTERVAL_DEFAULT ; +/* default to no multichannel (single server connection) */ + vol->multichannel = false; + vol->max_channels =1 ; +if (!mountdata) goto cifs_parse_mount_err;
@@ -1965,6 +1973,12 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,case Opt_rdma: vol->rdma = true;break ; +case Opt_multichannel: + vol->multichannel = true; +break ; +case Opt_nomultichannel: + vol->multichannel = false; +break ;case Opt_compress: vol->compression =UNKNOWN_TYPE ; cifs_dbg(VFS,
@@ -2128,6 +2142,15 @@ cifs_parse_mount_options(const char *mountdata, const char *devname, } vol->max_credits = option;break ; +case Opt_max_channels: +if (get_option_ul (args, &option) || option <1 || + option >CIFS_MAX_CHANNELS ) { +cifs_dbg (VFS ,"%s: Invalid max_channels value, needs to be 1-%d\n" , + __func__,CIFS_MAX_CHANNELS ); + goto cifs_parse_mount_err; + } + vol->max_channels = option; +break ;/* String Arguments */
@@ -2772,7 +2795,11 @@ cifs_get_tcp_session(struct smb_vol *volume_info) sizeof(tcp_ses->srcaddr));memcpy (&tcp_ses->dstaddr, &volume_info->dstaddr,sizeof (tcp_ses->dstaddr)); -generate_random_uuid (tcp_ses->client_guid); +if (volume_info->use_client_guid) +memcpy (tcp_ses->client_guid, volume_info->client_guid, +SMB2_CLIENT_GUID_SIZE ); +else +generate_random_uuid (tcp_ses->client_guid);/* * at this point we are the only ones with the pointer * to the struct since the kernel thread not created yet
@@ -2861,6 +2888,13 @@ static int match_session(struct cifs_ses *ses, struct smb_vol *vol) vol->sectype != ses->sectype)return 0 ; +/* +* If an existing session is limited to less channels than +* requested, it should not be reused +*/ +if (ses->chan_max < vol->max_channels) +return 0 ; +switch (ses->sectype) {case Kerberos:if (!uid_eq (vol->cred_uid, ses->cred_uid))
Signature · 1 line
2.16.4
PATCH v4 3/6 cifs: add server param
In reply to Aurelien AptelAs we get down to the transport layer, plenty of functions are passed the session pointer and assume the transport to use is ses->server.
Instead we modify those functions to pass (ses, server) so that we can decouple the session from the server.
--- fs/cifs/cifsglob.h | 3 ++- fs/cifs/cifsproto.h | 1 + fs/cifs/smb2proto.h | 3 ++- fs/cifs/smb2transport.c | 27 ++++++++++++++------------- fs/cifs/transport.c | 5 +++-- 5 files changed, 22 insertions(+), 17 deletions(-)
fs/cifs/cifsglob.h +2 -1
@@ -230,7 +230,8 @@ struct smb_version_operations {bool (*compare_fids)(struct cifsFileInfo *,struct cifsFileInfo *);/* setup request: allocate mid, sign message */ struct mid_q_entry *(*setup_request)(struct cifs_ses *, -struct smb_rqst *); +struct TCP_Server_Info *, +struct smb_rqst *);/* setup async request: allocate mid, sign message */ struct mid_q_entry *(*setup_async_request)(struct TCP_Server_Info *,struct smb_rqst *);
fs/cifs/cifsproto.h +1 -0
@@ -109,6 +109,7 @@ extern int SendReceive(const unsigned int /* xid */ , struct cifs_ses *,extern int SendReceiveNoRsp (const unsigned int xid,struct cifs_ses *ses,char *in_buf,int flags);extern struct mid_q_entry *cifs_setup_request (struct cifs_ses *, +struct TCP_Server_Info *,struct smb_rqst *);extern struct mid_q_entry *cifs_setup_async_request (struct TCP_Server_Info *,struct smb_rqst *);
fs/cifs/smb2proto.h +2 -1
@@ -46,7 +46,8 @@ extern int smb2_verify_signature(struct smb_rqst *, struct TCP_Server_Info *);extern int smb2_check_receive (struct mid_q_entry *mid,struct TCP_Server_Info *server,bool log_error);extern struct mid_q_entry *smb2_setup_request (struct cifs_ses *ses, -struct smb_rqst *rqst); +struct TCP_Server_Info *, +struct smb_rqst *rqst);extern struct mid_q_entry *smb2_setup_async_request (struct TCP_Server_Info *server,struct smb_rqst *rqst); extern struct cifs_ses *smb2_find_smb_ses(struct TCP_Server_Info *server,
fs/cifs/smb2transport.c +14 -13
@@ -610,18 +610,18 @@ smb2_mid_entry_alloc(const struct smb2_sync_hdr *shdr, }static int -smb2_get_mid_entry (struct cifs_ses *ses,struct smb2_sync_hdr *shdr, -struct mid_q_entry **mid) +smb2_get_mid_entry (struct cifs_ses *ses,struct TCP_Server_Info *server, +struct smb2_sync_hdr *shdr,struct mid_q_entry **mid) { -if (ses->server->tcpStatus == CifsExiting) +if (server->tcpStatus == CifsExiting)return -ENOENT ; -if (ses->server->tcpStatus == CifsNeedReconnect) { +if (server->tcpStatus == CifsNeedReconnect) {cifs_dbg (FYI ,"tcp session dead - return to caller to retry\n" );return -EAGAIN ; } -if (ses->server->tcpStatus == CifsNeedNegotiate && +if (server->tcpStatus == CifsNeedNegotiate && shdr->Command !=SMB2_NEGOTIATE )return -EAGAIN ;
@@ -638,11 +638,11 @@ smb2_get_mid_entry(struct cifs_ses *ses, struct smb2_sync_hdr *shdr,/* else ok - we are shutting down the session */ } - *mid =smb2_mid_entry_alloc (shdr, ses->server); + *mid =smb2_mid_entry_alloc (shdr, server);if (*mid ==NULL )return -ENOMEM ;spin_lock (&GlobalMid_Lock); -list_add_tail (&(*mid)->qhead, &ses->server->pending_mid_q); +list_add_tail (&(*mid)->qhead, &server->pending_mid_q);spin_unlock (&GlobalMid_Lock);return 0 ;
@@ -675,24 +675,25 @@ smb2_check_receive(struct mid_q_entry *mid, struct TCP_Server_Info *server, }struct mid_q_entry * -smb2_setup_request (struct cifs_ses *ses,struct smb_rqst *rqst) +smb2_setup_request (struct cifs_ses *ses,struct TCP_Server_Info *server, +struct smb_rqst *rqst) {int rc;struct smb2_sync_hdr *shdr = (struct smb2_sync_hdr *)rqst->rq_iov[0 ].iov_base;struct mid_q_entry *mid; -smb2_seq_num_into_buf (ses->server, shdr); +smb2_seq_num_into_buf (server, shdr); - rc =smb2_get_mid_entry (ses, shdr, &mid); + rc =smb2_get_mid_entry (ses, server, shdr, &mid);if (rc) { -revert_current_mid_from_hdr (ses->server, shdr); +revert_current_mid_from_hdr (server, shdr);return ERR_PTR (rc); } - rc =smb2_sign_rqst (rqst, ses->server); + rc =smb2_sign_rqst (rqst, server);if (rc) { -revert_current_mid_from_hdr (ses->server, shdr); +revert_current_mid_from_hdr (server, shdr);cifs_delete_mid (mid);return ERR_PTR (rc); }
fs/cifs/transport.c +3 -2
@@ -930,7 +930,8 @@ cifs_check_receive(struct mid_q_entry *mid, struct TCP_Server_Info *server, }struct mid_q_entry * -cifs_setup_request (struct cifs_ses *ses,struct smb_rqst *rqst) +cifs_setup_request (struct cifs_ses *ses,struct TCP_Server_Info *ignored, +struct smb_rqst *rqst) {int rc;struct smb_hdr *hdr = (struct smb_hdr *)rqst->rq_iov[0 ].iov_base;
@@ -1047,7 +1048,7 @@ compound_send_recv(const unsigned int xid, struct cifs_ses *ses, }for (i =0 ; i < num_rqst; i++) { - midQ[i] = server->ops->setup_request (ses, &rqst[i]); + midQ[i] = server->ops->setup_request (ses, server, &rqst[i]);if (IS_ERR (midQ[i])) {revert_current_mid (server, i);for (j =0 ; j < i; j++)
Signature · 1 line
2.16.4
PATCH v4 4/6 cifs: switch servers depending on binding state
In reply to Aurelien AptelCurrently a lot of the code to initialize a connection & session uses the cifs_ses as input. But depending on if we are opening a new session or a new channel we need to use different server pointers.
Add a "binding" flag in cifs_ses and a helper function that returns the server ptr a session should use (only in the sess establishment code path).
--- fs/cifs/cifs_spnego.c | 2 +- fs/cifs/cifsglob.h | 10 ++++++++++ fs/cifs/connect.c | 4 ++-- fs/cifs/sess.c | 5 +++-- fs/cifs/smb2ops.c | 2 +- fs/cifs/smb2pdu.c | 31 ++++++++++++++++--------------- 6 files changed, 33 insertions(+), 21 deletions(-)
fs/cifs/cifs_spnego.c +1 -1
@@ -98,7 +98,7 @@ struct key_type cifs_spnego_key_type = {struct key *cifs_get_spnego_key (struct cifs_ses *sesInfo) { -struct TCP_Server_Info *server = sesInfo->server; +struct TCP_Server_Info *server =cifs_ses_server (sesInfo);struct sockaddr_in *sa = (struct sockaddr_in *) &server->dstaddr;struct sockaddr_in6 *sa6 = (struct sockaddr_in6 *) &server->dstaddr;char *description, *dp;
fs/cifs/cifsglob.h +10 -0
@@ -994,6 +994,7 @@ struct cifs_ses {bool sign;/* is signing required? */ bool need_reconnect:1; /* connection reset, uid now invalid */ bool domainAuto:1; + bool binding:1; /* are we binding the session? */__u16 session_flags;__u8 smb3signingkey[SMB3_SIGN_KEY_SIZE ];__u8 smb3encryptionkey[SMB3_SIGN_KEY_SIZE ];
@@ -1021,6 +1022,15 @@ struct cifs_ses {atomic_t chan_seq;/* round robin state */ }; +static inline +struct TCP_Server_Info *cifs_ses_server (struct cifs_ses *ses) +{ +if (ses->binding) +return ses->chans[ses->chan_count].server; +else +return ses->server; +} +static inline bool cap_unix (struct cifs_ses *ses) {
fs/cifs/connect.c +2 -2
@@ -5176,7 +5176,7 @@ int cifs_negotiate_protocol(const unsigned int xid, struct cifs_ses *ses) { int rc = 0; -struct TCP_Server_Info *server = ses->server; +struct TCP_Server_Info *server =cifs_ses_server (ses);if (!server->ops->need_neg || !server->ops->negotiate)return -ENOSYS ;
@@ -5203,7 +5203,7 @@ cifs_setup_session(const unsigned int xid, struct cifs_ses *ses, struct nls_table *nls_info) {int rc = -ENOSYS ; -struct TCP_Server_Info *server = ses->server; +struct TCP_Server_Info *server =cifs_ses_server (ses); ses->capabilities = server->capabilities;if (linuxExtEnabled ==0 )
fs/cifs/sess.c +3 -2
@@ -342,6 +342,7 @@ int decode_ntlmssp_challenge(char *bcc_ptr, int blob_len,void build_ntlmssp_negotiate_blob (unsigned char *pbuffer,struct cifs_ses *ses) { +struct TCP_Server_Info *server =cifs_ses_server (ses);NEGOTIATE_MESSAGE *sec_blob = (NEGOTIATE_MESSAGE *)pbuffer;__u32 flags;
@@ -354,9 +355,9 @@ void build_ntlmssp_negotiate_blob(unsigned char *pbuffer,NTLMSSP_NEGOTIATE_128 |NTLMSSP_NEGOTIATE_UNICODE |NTLMSSP_NEGOTIATE_NTLM |NTLMSSP_NEGOTIATE_EXTENDED_SEC |NTLMSSP_NEGOTIATE_SEAL ; -if (ses->server->sign) +if (server->sign) flags |=NTLMSSP_NEGOTIATE_SIGN ; -if (!ses->server->session_estab || ses->ntlmssp->sesskey_per_smbsess) +if (!server->session_estab || ses->ntlmssp->sesskey_per_smbsess) flags |=NTLMSSP_NEGOTIATE_KEY_XCH ; sec_blob->NegotiateFlags =cpu_to_le32 (flags);
fs/cifs/smb2ops.c +1 -1
@@ -316,7 +316,7 @@ smb2_negotiate(const unsigned int xid, struct cifs_ses *ses) {int rc; - ses->server->CurrentMid =0 ; +cifs_ses_server (ses)->CurrentMid =0 ; rc =SMB2_negotiate (xid, ses);/* BB we probably don't need to retry with modern servers */ if (rc == -EAGAIN )
fs/cifs/smb2pdu.c +16 -15
@@ -791,7 +791,7 @@ SMB2_negotiate(const unsigned int xid, struct cifs_ses *ses)struct kvec rsp_iov;int rc =0 ;int resp_buftype; -struct TCP_Server_Info *server = ses->server; +struct TCP_Server_Info *server =cifs_ses_server (ses);int blob_offset, blob_length;char *security_blob;int flags =CIFS_NEG_OP ;
@@ -813,7 +813,7 @@ SMB2_negotiate(const unsigned int xid, struct cifs_ses *ses)memset (server->preauth_sha_hash,0 ,SMB2_PREAUTH_HASH_SIZE );memset (ses->preauth_sha_hash,0 ,SMB2_PREAUTH_HASH_SIZE ); -if (strcmp (ses->server->vals->version_string, +if (strcmp (server->vals->version_string,SMB3ANY_VERSION_STRING ) ==0 ) { req->Dialects[0 ] =cpu_to_le16 (SMB30_PROT_ID ); req->Dialects[1 ] =cpu_to_le16 (SMB302_PROT_ID );
@@ -829,7 +829,7 @@ SMB2_negotiate(const unsigned int xid, struct cifs_ses *ses) total_len +=8 ; }else {/* otherwise send specific dialect */ - req->Dialects[0 ] =cpu_to_le16 (ses->server->vals->protocol_id); + req->Dialects[0 ] =cpu_to_le16 (server->vals->protocol_id); req->DialectCount =cpu_to_le16 (1 ); total_len +=2 ; }
@@ -1171,7 +1171,7 @@ SMB2_sess_alloc_buffer(struct SMB2_sess_data *sess_data)int rc;struct cifs_ses *ses = sess_data->ses;struct smb2_sess_setup_req *req; -struct TCP_Server_Info *server = ses->server; +struct TCP_Server_Info *server =cifs_ses_server (ses);unsigned int total_len; rc = smb2_plain_req_init(SMB2_SESSION_SETUP, NULL, (void **) &req,
@@ -1258,22 +1258,23 @@ SMB2_sess_establish_session(struct SMB2_sess_data *sess_data) {int rc =0 ;struct cifs_ses *ses = sess_data->ses; +struct TCP_Server_Info *server =cifs_ses_server (ses); -mutex_lock (&ses->server->srv_mutex); -if (ses->server->ops->generate_signingkey) { - rc = ses->server->ops->generate_signingkey (ses); +mutex_lock (&server->srv_mutex); +if (server->ops->generate_signingkey) { + rc = server->ops->generate_signingkey (ses);if (rc) {cifs_dbg (FYI ,"SMB3 session key generation failed\n" ); -mutex_unlock (&ses->server->srv_mutex); +mutex_unlock (&server->srv_mutex);return rc; } } -if (!ses->server->session_estab) { - ses->server->sequence_number =0x2 ; - ses->server->session_estab = true; +if (!server->session_estab) { + server->sequence_number =0x2 ; + server->session_estab = true; } -mutex_unlock (&ses->server->srv_mutex); +mutex_unlock (&server->srv_mutex);cifs_dbg (FYI ,"SMB2/3 session established successfully\n" );spin_lock (&GlobalMid_Lock);
@@ -1509,7 +1510,7 @@ SMB2_select_sec(struct cifs_ses *ses, struct SMB2_sess_data *sess_data) {int type; - type =smb2_select_sectype (ses->server, ses->sectype); + type =smb2_select_sectype (cifs_ses_server (ses), ses->sectype);cifs_dbg (FYI ,"sess setup type %d\n" , type);if (type == Unspecified) { cifs_dbg(VFS,
@@ -1537,7 +1538,7 @@ SMB2_sess_setup(const unsigned int xid, struct cifs_ses *ses, const struct nls_table *nls_cp) {int rc =0 ; -struct TCP_Server_Info *server = ses->server; +struct TCP_Server_Info *server =cifs_ses_server (ses);struct SMB2_sess_data *sess_data;cifs_dbg (FYI ,"Session Setup\n" );
@@ -1563,7 +1564,7 @@ SMB2_sess_setup(const unsigned int xid, struct cifs_ses *ses,/* * Initialize the session hash with the server one. */ -memcpy (ses->preauth_sha_hash, ses->server->preauth_sha_hash, +memcpy (ses->preauth_sha_hash, server->preauth_sha_hash,SMB2_PREAUTH_HASH_SIZE );while (sess_data->func)
Signature · 1 line
2.16.4
PATCH v4 5/6 cifs: try opening channels after mounting
In reply to Aurelien AptelAfter doing mount() sucessfully we call cifs_try_adding_channels() which will open as many channels as it can.
Channels are closed when the master session is closed.
The master connection becomes the first channel.
,-------------> global cifs_tcp_ses_list <-------------------------.
| |
'- TCP_Server_Info <--> TCP_Server_Info <--> TCP_Server_Info <-'
(master con) (chan#1 con) (chan#2 con)
| ^ ^ ^
v '--------------------|--------------------'
cifs_ses |
- chan_count = 3 | - chans[] ---------------------'
- smb3signingkey[]
(master signing key)
Note how channel connections don't have sessions. That's because cifs_ses can only be part of one linked list (list_head are internal to the elements).
For signing keys, each channel has its own signing key which must be used only after the channel has been bound. While it's binding it must use the master session signing key.
For encryption keys, since channel connections do not have sessions attached we must now find matching session by looping over all sessions in smb2_get_enc_key().
Each channel is opened like a regular server connection but at the session setup request step it must set the SMB2_SESSION_REQ_FLAG_BINDING flag and use the session id to bind to.
Finally, while sending in compound_send_recv() for requests that aren't negprot, ses-setup or binding related, use a channel by cycling through the available ones (round-robin).
--- fs/cifs/cifsglob.h | 16 ++++ fs/cifs/cifsproto.h | 7 ++ fs/cifs/connect.c | 48 ++++++++--- fs/cifs/sess.c | 213 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/cifs/smb2misc.c | 37 ++++++--- fs/cifs/smb2ops.c | 18 ++-- fs/cifs/smb2pdu.c | 78 +++++++++++------- fs/cifs/smb2transport.c | 138 ++++++++++++++++++++++++------- fs/cifs/transport.c | 13 ++- 9 files changed, 479 insertions(+), 89 deletions(-)
fs/cifs/cifsglob.h +16 -0
@@ -1001,6 +1001,8 @@ struct cifs_ses {__u8 smb3decryptionkey[SMB3_SIGN_KEY_SIZE ];__u8 preauth_sha_hash[SMB2_PREAUTH_HASH_SIZE ]; +__u8 binding_preauth_sha_hash[SMB2_PREAUTH_HASH_SIZE ]; +/* * Network interfaces available on the server this session is * connected to.
@@ -1022,6 +1024,20 @@ struct cifs_ses {atomic_t chan_seq;/* round robin state */ }; +/* +* When binding a new channel, we need to access the channel which isn't fully +* established yet (one past the established count) +*/ + +static inline +struct cifs_chan *cifs_ses_binding_channel (struct cifs_ses *ses) +{ +if (ses->binding) +return &ses->chans[ses->chan_count]; +else +return NULL ; +} +static inline struct TCP_Server_Info *cifs_ses_server (struct cifs_ses *ses) {
fs/cifs/cifsproto.h +7 -0
@@ -243,6 +243,7 @@ extern void cifs_add_pending_open_locked(struct cifs_fid *fid,struct tcon_link *tlink, struct cifs_pending_open *open);extern void cifs_del_pending_open (struct cifs_pending_open *open); +extern struct TCP_Server_Info *cifs_get_tcp_session (struct smb_vol *vol);extern void cifs_put_tcp_session (struct TCP_Server_Info *server,int from_reconnect);extern void cifs_put_tcon (struct cifs_tcon *tcon);
@@ -585,6 +586,12 @@ void cifs_free_hash(struct crypto_shash **shash, struct sdesc **sdesc);extern void rqst_page_get_length (struct smb_rqst *rqst,unsigned int page,unsigned int *len,unsigned int *offset); +int cifs_try_adding_channels (struct cifs_ses *ses); +int cifs_ses_add_channel (struct cifs_ses *ses, +struct cifs_server_iface *iface); +bool is_server_using_iface (struct TCP_Server_Info *server, +struct cifs_server_iface *iface); +bool is_ses_using_iface (struct cifs_ses *ses,struct cifs_server_iface *iface);void extract_unc_hostname (const char *unc,const char **h,size_t *len);int copy_path_name (char *dst,const char *src);
fs/cifs/connect.c +35 -13
89 lines not on this page · raw message
fs/cifs/sess.c +213 -0
219 lines not on this page · raw message
fs/cifs/smb2misc.c +26 -11
@@ -29,6 +29,7 @@#include "cifs_unicode.h" #include "smb2status.h" #include "smb2glob.h" +#include "nterr.h" static int check_smb2_hdr (struct smb2_sync_hdr *shdr,__u64 mid)
@@ -788,23 +789,37 @@ smb311_update_preauth_hash(struct cifs_ses *ses, struct kvec *iov, int nvec)int i, rc;struct sdesc *d;struct smb2_sync_hdr *hdr; +struct TCP_Server_Info *server =cifs_ses_server (ses); -if (ses->server->tcpStatus == CifsGood) { -/* skip non smb311 connections */ -if (ses->server->dialect !=SMB311_PROT_ID ) -return 0 ; + hdr = (struct smb2_sync_hdr *)iov[0 ].iov_base; +/* neg prot are always taken */ +if (hdr->Command ==SMB2_NEGOTIATE ) + goto ok; -/* skip last sess setup response */ - hdr = (struct smb2_sync_hdr *)iov[0 ].iov_base; -if (hdr->Flags &SMB2_FLAGS_SIGNED ) -return 0 ; - } +/* +* If we process a command which wasn't a negprot it means the +* neg prot was already done, so the server dialect was set +* and we can test it. Preauth requires 3.1.1 for now. +*/ +if (server->dialect !=SMB311_PROT_ID ) +return 0 ; + +if (hdr->Command !=SMB2_SESSION_SETUP ) +return 0 ; + +/* skip last sess setup response */ +if ((hdr->Flags &SMB2_FLAGS_SERVER_TO_REDIR ) + && (hdr->Status ==NT_STATUS_OK + || (hdr->Status != +cpu_to_le32 (NT_STATUS_MORE_PROCESSING_REQUIRED )))) +return 0 ; - rc =smb311_crypto_shash_allocate (ses->server); +ok: + rc =smb311_crypto_shash_allocate (server);if (rc)return rc; - d = ses->server->secmech.sdescsha512; + d = server->secmech.sdescsha512; rc =crypto_shash_init (&d->shash);if (rc) {cifs_dbg (VFS ,"%s: could not init sha512 shash\n" , __func__);
fs/cifs/smb2ops.c +10 -8
@@ -3609,14 +3609,16 @@ smb2_get_enc_key(struct TCP_Server_Info *server, __u64 ses_id, int enc, u8 *key)u8 *ses_enc_key;spin_lock (&cifs_tcp_ses_lock); -list_for_each_entry (ses, &server->smb_ses_list, smb_ses_list) { -if (ses->Suid != ses_id) -continue ; - ses_enc_key = enc ? ses->smb3encryptionkey : - ses->smb3decryptionkey; -memcpy (key, ses_enc_key,SMB3_SIGN_KEY_SIZE ); -spin_unlock (&cifs_tcp_ses_lock); -return 0 ; +list_for_each_entry (server, &cifs_tcp_ses_list, tcp_ses_list) { +list_for_each_entry (ses, &server->smb_ses_list, smb_ses_list) { +if (ses->Suid == ses_id) { + ses_enc_key = enc ? ses->smb3encryptionkey : + ses->smb3decryptionkey; +memcpy (key, ses_enc_key,SMB3_SIGN_KEY_SIZE ); +spin_unlock (&cifs_tcp_ses_lock); +return 0 ; + } + } }spin_unlock (&cifs_tcp_ses_lock);
fs/cifs/smb2pdu.c +50 -28
115 lines not on this page · raw message
fs/cifs/smb2transport.c +110 -28
181 lines not on this page · raw message
fs/cifs/transport.c +12 -1
@@ -1003,7 +1003,18 @@ compound_send_recv(const unsigned int xid, struct cifs_ses *ses,return -EIO ; } - server = ses->server; +if (!ses->binding) { +uint index =0 ; + +if (ses->chan_count >1 ) { + index = (uint )atomic_inc_return (&ses->chan_seq); + index %= ses->chan_count; + } + server = ses->chans[index].server; + }else { + server =cifs_ses_server (ses); + } +if (server->tcpStatus == CifsExiting)return -ENOENT ;
Signature · 1 line
2.16.4
PATCH v4 6/6 cifs: dump channel info in DebugData
In reply to Aurelien Aptel* show server&TCP states for extra channels
* mention if an interface has a channel connected to it
--- fs/cifs/cifs_debug.c | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-)
fs/cifs/cifs_debug.c +34 -1
@@ -121,6 +121,27 @@ static void cifs_debug_tcon(struct seq_file *m, struct cifs_tcon *tcon)seq_putc (m,'\n' ); } +static void +cifs_dump_channel (struct seq_file *m,int i,struct cifs_chan *chan) +{ +struct TCP_Server_Info *server = chan->server; + +seq_printf (m,"\t\tChannel %d Number of credits: %d Dialect 0x%x " +"TCP status: %d Instance: %d Local Users To Server: %d " +"SecMode: 0x%x Req On Wire: %d In Send: %d " +"In MaxReq Wait: %d\n" , + i+1 , + server->credits, + server->dialect, + server->tcpStatus, + server->reconnect_instance, + server->srv_count, + server->sec_mode, +in_flight (server), +atomic_read (&server->in_send), +atomic_read (&server->num_waiters)); +} +static void cifs_dump_iface (struct seq_file *m,struct cifs_server_iface *iface) {
@@ -377,6 +398,13 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)if (ses->sign)seq_puts (m," signed" ); +if (ses->chan_count >1 ) { +seq_printf (m,"\n\n\tExtra Channels: %lu\n" , + ses->chan_count-1 ); +for (j =1 ; j < ses->chan_count; j++) +cifs_dump_channel (m, j, &ses->chans[j]); + } +seq_puts (m,"\n\tShares:" ); j =0 ;
@@ -415,8 +443,13 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)seq_printf (m,"\n\tServer interfaces: %zu\n" , ses->iface_count);for (j =0 ; j < ses->iface_count; j++) { +struct cifs_server_iface *iface; + + iface = &ses->iface_list[j];seq_printf (m,"\t%d)" , j); -cifs_dump_iface (m, &ses->iface_list[j]); +cifs_dump_iface (m, iface); +if (is_ses_using_iface (ses, iface)) +seq_puts (m,"\t\t[CONNECTED]\n" ); }spin_unlock (&ses->iface_lock); }
Signature · 1 line
2.16.4