How to use SMB Direct
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in upstream. I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yes
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdma
It's failed because of "CIFS VFS: smbd_create_id:614 rdma_resolve_addr() completed -113" Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand
Signature · 8 lines
Best regards! XiaoLi Feng 冯小丽 Red Hat Software (Beijing) Co.,Ltd filesystem-qe Team IRC:xifeng,#channel: fs-qe Tel:+86-10-8388112 9/F, Raycom
Quoted 16 lines
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in upstream. I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yes
Quoted 5 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()completed -113"
Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24
Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 81 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 18 lines
-----Original Message-----
From: [email protected] <[email protected]> On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 6 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
ronnie sahlberg
reviewerI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.
For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).
Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.
Linux serverside SMBD support is not ready afaik.
Quoted 101 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=445Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 81 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 18 lines
-----Original Message-----
From: [email protected] <[email protected]> On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 6 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?
----- Original Message -----
Quoted 131 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.Linux serverside SMBD support is not ready afaik.
Quoted 106 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failedport=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 84 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 20 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in
upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 7 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
ronnie sahlberg
reviewerQuoted 3 lines · Xiaoli Feng
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?
Yeah, that should definitely work. I think Steve, or a colleague, has run through the same set of the tests in our buildbot (manually?) in that configuration.
If you can please test the work-in-progress cifsd kernel server. It should supposedly work with soft-rdma and if it does then that could be a way where we could
integrate SMBDirect testing in the buildbot.
I haven't had any time to research cifsd or set it up and test unfortunately :-(
regards
ronnie sahlberg
Quoted 134 lines
----- Original Message -----
Quoted 131 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.Linux serverside SMBD support is not ready afaik.
Quoted 106 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failedport=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 84 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 20 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in
upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 7 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
Hello Xiaoli,
cifsd is an in-kernel SMB server which supports SMB Direct. If you want to test SMB Direct with cifs, you can use cifsd.
Currently we have tested SMB Direct between two old fashioned Mellanox ConnectX devices which are connected directly, and between two soft RoCE devices in kernel.
You can get cifsd's code from the following url, and If you have any questions, we will help you.
https://github.com/cifsd-team/cifsd.git
Thank you.
2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <[email protected]>님이 작성:
Quoted 137 lines
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?
----- Original Message -----
Quoted 131 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.Linux serverside SMBD support is not ready afaik.
Quoted 106 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failedport=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 84 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 20 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in
upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 7 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
ronnie sahlberg
reviewerQuoted 9 lines · Hyeoncheol Lee
Hello Xiaoli,
cifsd is an in-kernel SMB server which supports SMB Direct. If you want to test SMB Direct with cifs, you can use cifsd.
Currently we have tested SMB Direct between two old fashioned Mellanox ConnectX devices which are connected directly, and between two soft RoCE devices in kernel.
Thanks Hyeoncheol,
This is very interesting!
We have a buildbot upstream for the cifs client where we test every set of patches before they go to linus :
http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/302
I havent had time to test with your cifsd yet but as you say you can use soft-RoCE
you are saying it will work doing soft-RoCE between one VM running cifs.ko and a basic virtio NIC
to another VM running your cifsd and also using a basic virtio NIC?
I would really want to set up so we can do RDMA tests for cifs.ko, and by accident cifsd, in our buildbot.
regards
ronnie sahlberg
Quoted 147 lines
You can get cifsd's code from the following url, and If you have any questions, we will help you.
https://github.com/cifsd-team/cifsd.git
Thank you.
2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <[email protected]>님이 작성:
Quoted 137 lines
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?
----- Original Message -----
Quoted 131 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.Linux serverside SMBD support is not ready afaik.
Quoted 106 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failedport=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 84 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 20 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in
upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 7 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
Hello Ronnie,
2019년 12월 19일 (목) 오후 3:57, ronnie sahlberg <[email protected]>님이 작성:
Quoted 20 lines · Hyeoncheol Lee
Quoted 9 lines
Hello Xiaoli,
cifsd is an in-kernel SMB server which supports SMB Direct. If you want to test SMB Direct with cifs, you can use cifsd.
Currently we have tested SMB Direct between two old fashioned Mellanox ConnectX devices which are connected directly, and between two soft RoCE devices in kernel.
Thanks Hyeoncheol,
This is very interesting!
We have a buildbot upstream for the cifs client where we test every set of patches before they go to linus :
http://smb3-test-rhel-75.southcentralus.cloudapp.azure.com/#/builders/2/builds/302
Quoted 5 lines
I havent had time to test with your cifsd yet but as you say you can use soft-RoCE
you are saying it will work doing soft-RoCE between one VM running cifs.ko and a basic virtio NIC
to another VM running your cifsd and also using a basic virtio NIC?
Yes, I had tested SMB Direct between the two VMs that are configured as you said.
Quoted 4 lines
I would really want to set up so we can do RDMA tests for cifs.ko, and by accident cifsd, in our buildbot.
We will help you. but before that, our cifsd's xfstests test-cases are a different
with your cifs' test-cases. your some test-cases may be failed for cifsd. our test-cases were suggested by Steve in the past.
So we want to know the list of your test-cases and check that the cases will succeed for cifsd.
And while we check the test-cases, the common test-cases between cifs and cifsd can be tested in the buildbot.
Thank you.
Quoted 152 lines
regards
ronnie sahlbergQuoted 147 lines
You can get cifsd's code from the following url, and If you have any questions, we will help you.
https://github.com/cifsd-team/cifsd.git
Thank you.
2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <[email protected]>님이 작성:
Quoted 137 lines
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?
----- Original Message -----
Quoted 131 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.Linux serverside SMBD support is not ready afaik.
Quoted 106 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failedport=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 84 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 20 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in
upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 7 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
Please read README file to build cifsd
And SMB direct disabled by default.
Enable smb direct support on config.
<M> CIFS server support
[*] Support for SMB Direct protocol
Quoted 172 lines
-----Original Message-----
From: Hyeoncheol Lee <[email protected]>
Sent: Thursday, December 19, 2019 3:48 PM
To: Xiaoli Feng <[email protected]>
Cc: linux-cifs <[email protected]>; Tom Talpey <[email protected]>; [email protected]; ronnie sahlberg <[email protected]>
Subject: Re: [Linux-cifsd-devel] How to use SMB DirectHello Xiaoli,
cifsd is an in-kernel SMB server which supports SMB Direct. If you want to test SMB Direct with cifs, you can use cifsd.
Currently we have tested SMB Direct between two old fashioned Mellanox ConnectX devices which are connected directly, and between two soft RoCE devices in kernel.
You can get cifsd's code from the following url, and If you have any questions, we will help you.
https://protect2.fireeye.com/url?k=02be8777-5f6a3b1f-02bf0c38-
0cc47a3356b2-b295524a9bc31dd5&u=https://github.com/cifsd-team/cifsd.gitThank you.
2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <[email protected]>님이 작성:
Quoted 3 lines
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file server(has rdma hardware) with SMB Direct in
linux client(has rdma hardware)?
Quoted 82 lines
----- Original Message -----
Quoted 80 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).
Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be limited to just using real hw for any tests.
Linux serverside SMBD support is not ready afaik.
Quoted 58 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connectfailed
port=445Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 35 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>,
[email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 19 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works in upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 8 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+ kernel.
mount //$RDMA/cifs cifs -o
user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address
issue.
Quoted 41 lines
Quoted 41 lines
Quoted 41 lines
Quoted 41 lines
Tom.
Quoted 38 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Quoted 10 lines
Quoted 9 lines
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Red Hat Software (Beijing) Co.,Ltd filesystem-qe Team IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom_______________________________________________
Linux-cifsd-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-cifsd-devel
Hello Hyeoncheol,
When use the cifsd you provide, are there any required for Samba server? Now I have two machines that have Mellanox ConnectX-3. And install 5.5.0-rc1+. If one machine install the cifsd. Then I can use the SMB Direct without the concern of Samba server. Is it right?
As I know, seems there are two projects for SMB Direct(kernel part). One is the cifsd. The others is Metze's repo. Which one is planning to go into linux tree or both are?
https://github.com/cifsd-team/cifsd.git
https://git.samba.org/?p=metze/linux/smbdirect.git;a=shortlog;h=refs/heads/smbdirect-work-in-progress
Thanks.
----- Original Message -----
Quoted 171 lines
From: "Hyeoncheol Lee" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "ronnie sahlberg" <[email protected]>, "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>, [email protected]
Sent: Thursday, December 19, 2019 2:47:40 PM
Subject: Re: How to use SMB DirectHello Xiaoli,
cifsd is an in-kernel SMB server which supports SMB Direct. If you want to test SMB Direct with cifs, you can use cifsd.
Currently we have tested SMB Direct between two old fashioned Mellanox ConnectX devices which are connected directly, and between two soft RoCE devices in kernel.
You can get cifsd's code from the following url, and If you have any questions, we will help you.
https://github.com/cifsd-team/cifsd.git
Thank you.
2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <[email protected]>님이 작성:
Quoted 144 lines
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file
server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?----- Original Message -----
Quoted 136 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.Linux serverside SMBD support is not ready afaik.
Quoted 110 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failedport=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 88 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 22 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works
in
upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 8 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+
kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
The work of Metze's on RDMA/smbdirect extensions to Samba, rely in
part on kernel code. I can help merge that into the kernel but I
would like some agreement between Metze and Long Li (and Tom etc.) on what would be useful to be shared between cifs.ko and the proposed
RDMA helpe driver for Samba. From discussions this fall, it is
probably too early to consider the merge request, but I am very open to following up discussions on this.
The alternative RDMA/smbdirect implementation in cifsd (the proposed smb3 kernel server written by Namjae and other) is very exciting and an interesting topic, but likely unrelated to the Samba server
implementation that Metze is working on. The issues involved in
merging cifsd (the SMB3 kernel server) into the mainline kernel are quite different than those related to Samba's RDMA implementation, and instead mostly have to deal with addressing review comments from others (I have sent namjae various suggestions which he appears to be
addressing). More review feedback of the cifsd kernel server will be
very helpful - but the main issues are not RDMA related.
Quoted 190 lines · Xiaoli Feng
Hello Hyeoncheol,
When use the cifsd you provide, are there any required for Samba server? Now I have two machines that have Mellanox ConnectX-3. And install 5.5.0-rc1+. If one machine install the cifsd. Then I can use the SMB Direct without the concern of Samba server. Is it right?
As I know, seems there are two projects for SMB Direct(kernel part). One is the cifsd. The others is Metze's repo. Which one is planning to go into linux tree or both are?
https://github.com/cifsd-team/cifsd.git
https://git.samba.org/?p=metze/linux/smbdirect.git;a=shortlog;h=refs/heads/smbdirect-work-in-progressThanks.
----- Original Message -----
Quoted 171 lines
From: "Hyeoncheol Lee" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "ronnie sahlberg" <[email protected]>, "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>, [email protected]
Sent: Thursday, December 19, 2019 2:47:40 PM
Subject: Re: How to use SMB DirectHello Xiaoli,
cifsd is an in-kernel SMB server which supports SMB Direct. If you want to test SMB Direct with cifs, you can use cifsd.
Currently we have tested SMB Direct between two old fashioned Mellanox ConnectX devices which are connected directly, and between two soft RoCE devices in kernel.
You can get cifsd's code from the following url, and If you have any questions, we will help you.
https://github.com/cifsd-team/cifsd.git
Thank you.
2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <[email protected]>님이 작성:
Quoted 144 lines
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file
server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?----- Original Message -----
Quoted 136 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.Linux serverside SMBD support is not ready afaik.
Quoted 110 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failedport=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 88 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 22 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works
in
upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 8 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+
kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom
Signature · 3 lines
Thanks, Steve
Am 20.12.19 um 04:26 schrieb Steve French:
Quoted 7 lines
The work of Metze's on RDMA/smbdirect extensions to Samba, rely in
part on kernel code. I can help merge that into the kernel but I
would like some agreement between Metze and Long Li (and Tom etc.) on what would be useful to be shared between cifs.ko and the proposed
RDMA helpe driver for Samba. From discussions this fall, it is
probably too early to consider the merge request, but I am very open to following up discussions on this.
Yes, it's too early. Sadly I still don't have funding to drive this further. But once I have time for it again I'll try to get this to a state where we no longer have memory leaks and pass the microsoft testsuite and manual testing. Once that's done it should be pretty easy to plug this also into cifs.ko (I already have patches and it worked basically a few month ago) and also into the cifsd code. So that we can compare how the implementations perform and if mine is able to perform in a similar way we can start to think about merging it to the kernel have just one implementation for the SMB-Direct protocol that is usable for all 4 cases (cifs.ko and smbclient as client and Samba's smbd and the kernel cifsd as server).
metze
Hello Xiaoli,
2019년 12월 20일 (금) 오후 12:00, Xiaoli Feng <[email protected]>님이 작성:
Quoted 8 lines
Hello Hyeoncheol,
When use the cifsd you provide, are there any required for Samba server? Now I have two machines that have Mellanox ConnectX-3. And install 5.5.0-rc1+. If one machine install the cifsd. Then I can use the SMB Direct without the concern of Samba server. Is it right?
Yes, you can use the SMB Direct without Samba server.
Following are instructions for cifsd setup.
And cifsd utils and a detailed description can be found in
https://github.com/cifsd-team/cifsd-tools
- Build and install cifsd kernel driver
modprobe cifsd
- Create user/password for SMB share cifsuseradd -a <Enter USERNAME for SMB share access> Enter password for SMB share access
- Create /etc/cifs/smb.conf file, add SMB share in smb.conf file Refer smb.conf.example
- Start cifsd user space daemon
cifsd
- Access share from Windows or Linux using CIFS
Thank you.
Quoted 182 lines
As I know, seems there are two projects for SMB Direct(kernel part). One is the cifsd. The others is Metze's repo. Which one is planning to go into linux tree or both are?
https://github.com/cifsd-team/cifsd.git
https://git.samba.org/?p=metze/linux/smbdirect.git;a=shortlog;h=refs/heads/smbdirect-work-in-progressThanks.
----- Original Message -----
Quoted 171 lines
From: "Hyeoncheol Lee" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "ronnie sahlberg" <[email protected]>, "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>, [email protected]
Sent: Thursday, December 19, 2019 2:47:40 PM
Subject: Re: How to use SMB DirectHello Xiaoli,
cifsd is an in-kernel SMB server which supports SMB Direct. If you want to test SMB Direct with cifs, you can use cifsd.
Currently we have tested SMB Direct between two old fashioned Mellanox ConnectX devices which are connected directly, and between two soft RoCE devices in kernel.
You can get cifsd's code from the following url, and If you have any questions, we will help you.
https://github.com/cifsd-team/cifsd.git
Thank you.
2019년 12월 19일 (목) 오전 11:35, Xiaoli Feng <[email protected]>님이 작성:
Quoted 144 lines
Thanks Ronnie for the information. Last question: Is it supported that mount the windows file
server(has rdma hardware) with SMB Direct in linux client(has rdma hardware)?----- Original Message -----
Quoted 136 lines
From: "ronnie sahlberg" <[email protected]>
To: "Xiaoli Feng" <[email protected]>
Cc: "Tom Talpey" <[email protected]>, "linux-cifs" <[email protected]>
Sent: Wednesday, December 18, 2019 4:12:32 PM
Subject: Re: How to use SMB DirectI don't think samba supports SMB Direct.
Metze used to have a private repo with some experimental patches but I don't think it
has landed in official samba yet.For a linux server, there is an experimental kernel based server called cifsd which should support SMBDirect
but is incomplete in other areas. It is not part of linus tree yet (but soon I hope).Windows servers support SMBDirect but I am not aware of any soft-rdma support so you might be
limited to just using real hw for any tests.Linux serverside SMBD support is not ready afaik.
Quoted 110 lines · Xiaoli Feng
Hello Tom,
Then I try to use IP to mount, it show this error:
[79912.177783] CIFS VFS: _smbd_get_connection:1740 rdma_connect failed port=5445 [79912.220723] CIFS VFS: _smbd_get_connection:1740 rdma_connect failedport=445
Client:
$ mount //172.31.0.250/cifs cifs -o user=root,password=redhat,rdma mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)Server:
$ ib addr |grep 172.31.0.250/ mlx4_ib0: Link UP, Interface UP 172.31.0.250/24Samba version is 4.11.2. And firewalld is stopped.
----- Original Message -----
Quoted 88 lines
From: "Tom Talpey" <[email protected]>
To: "Xiaoli Feng" <[email protected]>, [email protected]
Sent: Tuesday, December 17, 2019 1:11:24 AM
Subject: RE: How to use SMB DirectQuoted 22 lines
-----Original Message-----
From: [email protected]
<[email protected]>
On
Behalf Of Xiaoli Feng
Sent: Monday, December 16, 2019 10:30 AM
To: [email protected]
Subject: [EXTERNAL] How to use SMB DirectHello guys,
I'd like to test SMB Direct. But it's failed. I'm not sure if it works
in
upstream.
I setup samba server on one rdma machine with 5.5.0-rc1+ kernel. The
smb.conf is:
[cifs]
path=/mnt/cifs
writeable=yesQuoted 8 lines
Then I try to mount the share on another rdma machine with 5.5.0-rc1+
kernel.
mount //$RDMA/cifs cifs -o user=root,password=$password,rdmaIt's failed because of "CIFS VFS: smbd_create_id:614
rdma_resolve_addr()
completed -113"Errno 113 is "no route to host". Sounds like a network or address issue.
Tom.
Quoted 45 lines
Does SMB Direct work fine in upstream?
Thanks.
$ cat /boot/config-5.5.0-rc1+ |grep SMB_DIRECT CONFIG_CIFS_SMB_DIRECT=y $ ibstat CA 'mlx4_0' CA type: MT4099 Number of ports: 2 Firmware version: 2.42.5000 Hardware version: 1 Node GUID: 0xf4521403007be0e0 System image GUID: 0xf4521403007be0e3 Port 1: State: Active Physical state: LinkUp Rate: 56 Base lid: 29 LMC: 0 SM lid: 1 Capability mask: 0x0259486a Port GUID: 0xf4521403007be0e1 Link layer: InfiniBand Port 2: State: Active Physical state: LinkUp Rate: 40 Base lid: 44 LMC: 1 SM lid: 36 Capability mask: 0x02594868 Port GUID: 0xf4521403007be0e2 Link layer: InfiniBand--
Best regards!
XiaoLi Feng 冯小丽Red Hat Software (Beijing) Co.,Ltd
filesystem-qe Team
IRC:xifeng,#channel: fs-qe
Tel:+86-10-8388112
9/F, Raycom