Linux kernel SMB server (CIFSD)
From: Sergey Senozhatsky <[email protected]>
[ Note: The following abstract was submitted via the Linsux Plumbers
Conference website. Per the instructions that were posted for the Maintainer's / Kernel Summit Call for Proposals[1], the proposal should also be posted on the ksummit-discuss list, so that people can comment on the proposal, and perhaps start a discussion before the summit.
[1] https://lwn.net/Articles/788378/
Please note that topic proposals for both the Kernel Summit and the Maintainer's Summit are still welcome, and the deadline has been extended to June 3rd. -- Ted ]
CIFSD is a new SMB server implementation for the Linux kernel (out of tree, as of this moment), intended to be minimalistic, yet providing better performance than user-space analogues. This talk will briefly introduce CIFSD, its architecture (high level) and its main design goals: outline components, their responsibilities and communication protocols.
The work is still in progress, we are in pretty early stages, so in the second part of the talk we will focus on current state of affairs and our future development plans - we are looking at possibility to merge CIFSD with the upstream Linux kernel, sometime in the future.
Cc-ing Jan, Amir, Jeff
Hi!
Quoted 1 line · Theodore Ts'o
From: Sergey Senozhatsky <[email protected]>
[..]
Quoted 7 lines
CIFSD is a new SMB server implementation for the Linux kernel (out of tree, as of this moment), intended to be minimalistic, yet providing better performance than user-space analogues. This talk will briefly introduce CIFSD, its architecture (high level) and its main design goals: outline components, their responsibilities and communication protocols.
Quoted 5 lines
The work is still in progress, we are in pretty early stages, so in the second part of the talk we will focus on current state of affairs and our future development plans - we are looking at possibility to merge CIFSD with the upstream Linux kernel, sometime in the future.
To extend the topic with some discussion points:
- We are facing a rather familiar problem. Basically, SMB2 CHANGE_NOTIFY
[1] is something what fsnotify normally does, except that we can't use it in cifsd kernel module. I see that NFS guys had [2] same issues some time ago.
So the question is - how to do fs notify style monitoring in a kernel module (nfsd, cifsd)? Any chance the kernel can start exporting fsnotify symbols?
[1] https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/598f395a-e7a2-4cc8-afb3-ccb30dd2df7c
[2] https://lore.kernel.org/linux-fsdevel/[email protected]/
-ss
Quoted 2 lines · Sergey Senozhatsky
Quoted 1 line · Theodore Ts'o
From: Sergey Senozhatsky <[email protected]>
[..]
Quoted 10 lines
To extend the topic with some discussion points:
- We are facing a rather familiar problem. Basically, SMB2
CHANGE_NOTIFY
[1] is something what fsnotify normally does, except that we can't use it in cifsd kernel module. I see that NFS guys had [2] same issues some time ago.So the question is - how to do fs notify style monitoring in a kernel module (nfsd, cifsd)? Any chance the kernel can start exporting fsnotify symbols?
I even looked at LSM hooks, because they do what we need - invoke creat, open, truncate, chmod, chown, unlink, callbacks. But, first, I do realize that LSM was not meant to be used as some sort of fsnotify replacement, and, second, LSM symbols are not exported anyway :)
-ss
Quoted 7 lines · Sergey Senozhatsky
Quoted 2 lines · Sergey Senozhatsky
Quoted 1 line · Theodore Ts'o
From: Sergey Senozhatsky <[email protected]>
[..]
Quoted 2 lines
To extend the topic with some discussion points:
Quoted 8 lines
Quoted 8 lines
- We are facing a rather familiar problem. Basically, SMB2
CHANGE_NOTIFY
[1] is something what fsnotify normally does, except that we can't use it in cifsd kernel module. I see that NFS guys had [2] same issues some time ago.So the question is - how to do fs notify style monitoring in a kernel module (nfsd, cifsd)? Any chance the kernel can start exporting fsnotify symbols?
I don't see a problem with exporting those symbols.
As a matter of fact, I have a plan to use them also from overlayfs. At the time that Jeff posted his patches, there was no active fsnotify maintainer.
Jeff, did you abandon this effort for nfsd?
Quoted 6 lines
I even looked at LSM hooks, because they do what we need - invoke creat, open, truncate, chmod, chown, unlink, callbacks. But, first, I do realize that LSM was not meant to be used as some sort of fsnotify replacement, and, second, LSM symbols are not exported anyway :)
fsnotify could actually be called from LSM hooks, see:
https://github.com/amir73il/linux/commits/fsnotify_dirent_perm
It may or may not be clear to you, but what you get extra from fsnotify compared to using bare LSM hooks is:
- Minimal performance overhead for a non-marked object
- Ability to subscribe/unsubscribe to certain events dynamically
- Manage multiple subscriber instances (groups), like one per SMB session
- Queuing infrastructure for async events if you need it
Thanks,
Amir.
Quoted 17 lines · Amir Goldstein
Quoted 6 lines · Sergey Senozhatsky
Quoted 2 lines · Sergey Senozhatsky
Quoted 1 line · Theodore Ts'o
From: Sergey Senozhatsky <[email protected]>
[..]
Quoted 2 lines
To extend the topic with some discussion points:
Quoted 8 lines
Quoted 8 lines
- We are facing a rather familiar problem. Basically, SMB2
CHANGE_NOTIFY
[1] is something what fsnotify normally does, except that we can't use it in cifsd kernel module. I see that NFS guys had [2] same issues some time ago.So the question is - how to do fs notify style monitoring in a kernel module (nfsd, cifsd)? Any chance the kernel can start exporting fsnotify symbols?
Quoted 6 lines
I don't see a problem with exporting those symbols.
As a matter of fact, I have a plan to use them also from overlayfs. At the time that Jeff posted his patches, there was no active fsnotify maintainer.
Jeff, did you abandon this effort for nfsd?
(cc'ing Trond)
Trond has been driving those patches more recently. He posted a revised set of them recently, and they still use fsnotify to detect unlink activity:
https://www.spinics.net/lists/linux-nfs/msg73692.html
Quoted 15 lines
Quoted 5 lines
I even looked at LSM hooks, because they do what we need - invoke creat, open, truncate, chmod, chown, unlink, callbacks. But, first, I do realize that LSM was not meant to be used as some sort of fsnotify replacement, and, second, LSM symbols are not exported anyway :)
fsnotify could actually be called from LSM hooks, see:
https://github.com/amir73il/linux/commits/fsnotify_dirent_permIt may or may not be clear to you, but what you get extra from fsnotify compared to using bare LSM hooks is:
- Minimal performance overhead for a non-marked object
- Ability to subscribe/unsubscribe to certain events dynamically
- Manage multiple subscriber instances (groups), like one per SMB session
- Queuing infrastructure for async events if you need it
Signature · 1 line
Jeff Layton <[email protected]>
On (07/24/19 13:50), Amir Goldstein wrote:
[..]
Quoted 4 lines
I don't see a problem with exporting those symbols.
As a matter of fact, I have a plan to use them also from overlayfs. At the time that Jeff posted his patches, there was no active fsnotify maintainer.
Cool.
Quoted 9 lines
Jeff, did you abandon this effort for nfsd?
Quoted 6 lines
I even looked at LSM hooks, because they do what we need - invoke creat, open, truncate, chmod, chown, unlink, callbacks. But, first, I do realize that LSM was not meant to be used as some sort of fsnotify replacement, and, second, LSM symbols are not exported anyway :)
Quoted 5 lines
fsnotify could actually be called from LSM hooks, see:
https://github.com/amir73il/linux/commits/fsnotify_dirent_permIt may or may not be clear to you, but what you get extra from fsnotify compared to using bare LSM hooks is:
Oh yeah, LSM does not really fit. I looked at it just out of curiosity.
Quoted 2 lines
- Minimal performance overhead for a non-marked object
- Ability to subscribe/unsubscribe to certain events dynamically
Absolutely, we need to "subscribe/unsubscribe dynamically".
Quoted 1 line
- Manage multiple subscriber instances (groups), like one per SMB session
We need this as well, you're right.
Quoted 1 line
- Queuing infrastructure for async events if you need it
-ss
Quoted 34 lines · Amir Goldstein
Quoted 7 lines · Sergey Senozhatsky
Quoted 2 lines · Sergey Senozhatsky
Quoted 1 line · Theodore Ts'o
From: Sergey Senozhatsky <[email protected]>
[..]
Quoted 2 lines
To extend the topic with some discussion points:
Quoted 8 lines
Quoted 8 lines
- We are facing a rather familiar problem. Basically, SMB2
CHANGE_NOTIFY
[1] is something what fsnotify normally does, except that we can't use it in cifsd kernel module. I see that NFS guys had [2] same issues some time ago.So the question is - how to do fs notify style monitoring in a kernel module (nfsd, cifsd)? Any chance the kernel can start exporting fsnotify symbols?
I don't see a problem with exporting those symbols.
As a matter of fact, I have a plan to use them also from overlayfs. At the time that Jeff posted his patches, there was no active fsnotify maintainer.
Jeff, did you abandon this effort for nfsd?Quoted 6 lines
I even looked at LSM hooks, because they do what we need - invoke creat, open, truncate, chmod, chown, unlink, callbacks. But, first, I do realize that LSM was not meant to be used as some sort of fsnotify replacement, and, second, LSM symbols are not exported anyway :)
fsnotify could actually be called from LSM hooks, see:
https://github.com/amir73il/linux/commits/fsnotify_dirent_perm
Quoted 6 lines
It may or may not be clear to you, but what you get extra from fsnotify compared to using bare LSM hooks is:
- Minimal performance overhead for a non-marked object
- Ability to subscribe/unsubscribe to certain events dynamically
- Manage multiple subscriber instances (groups), like one per SMB session
- Queuing infrastructure for async events if you need it
.... and then there's the sad fact that LSM hooks have no well-defined semantics and their coverage is... suboptimal, to put it very mildly.
Quoted 7 lines · Jeff Layton
(cc'ing Trond)
Trond has been driving those patches more recently. He posted a revised set of them recently, and they still use fsnotify to detect unlink activity:
Jan, do you have objections?
Would be great to have fsnotify export patch in linux-next, maybe.
-ss
Quoted 25 lines · Jeff Layton
Quoted 17 lines · Amir Goldstein
Quoted 6 lines · Sergey Senozhatsky
Quoted 2 lines · Sergey Senozhatsky
Quoted 1 line · Theodore Ts'o
From: Sergey Senozhatsky <[email protected]>
[..]
Quoted 2 lines
To extend the topic with some discussion points:
Quoted 8 lines
Quoted 8 lines
- We are facing a rather familiar problem. Basically, SMB2
CHANGE_NOTIFY
[1] is something what fsnotify normally does, except that we can't use it in cifsd kernel module. I see that NFS guys had [2] same issues some time ago.So the question is - how to do fs notify style monitoring in a kernel module (nfsd, cifsd)? Any chance the kernel can start exporting fsnotify symbols?
Quoted 6 lines
I don't see a problem with exporting those symbols.
As a matter of fact, I have a plan to use them also from overlayfs. At the time that Jeff posted his patches, there was no active fsnotify maintainer.
Jeff, did you abandon this effort for nfsd?
Quoted 7 lines
(cc'ing Trond)
Trond has been driving those patches more recently. He posted a revised set of them recently, and they still use fsnotify to detect unlink activity:
The fsnotify export bits look sane to me except perhaps fsnotify_put_group() and fsnotify_wait_marks_destroyed(). Their usage in "[PATCH 05/16] nfsd: add a new struct file caching facility to nfsd" looks buggy AFAICT. One is supposed to destroy a group with fsnotify_destroy_group() instead of just dropping the group reference (that does just the final cleanup but expects fsnotify_destroy_group() has been already called).
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 11 lines · Sergey Senozhatsky
Quoted 7 lines · Jeff Layton
(cc'ing Trond)
Trond has been driving those patches more recently. He posted a revised set of them recently, and they still use fsnotify to detect unlink activity:
Jan, do you have objections?
Would be great to have fsnotify export patch in linux-next, maybe.
I have no objections to export bits of fsnotify to modules. After all kernel/audit_tree.c is another external user of the fsnotify bits, just it happens not to be modularized so there was never need for exports.
Once NFS guys (or someone else) have something that is close to inclusion, I can push the export patch to my tree (just CC me, Amir, and fsdevel on the series). I prefer not to export symbols for something that is out of tree and will not be getting upstream in a near future. After all the export patch is easy to carry in your tree if you need it.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 8 lines · Jan Kara
Quoted 7 lines · Sergey Senozhatsky
Quoted 6 lines · Jeff Layton
(cc'ing Trond)
Trond has been driving those patches more recently. He posted a revised set of them recently, and they still use fsnotify to detect unlink activity:
Quoted 8 lines
Quoted 4 lines
Jan, do you have objections?
Would be great to have fsnotify export patch in linux-next, maybe.I have no objections to export bits of fsnotify to modules. After all
kernel/audit_tree.cis another external user of the fsnotify bits, just it happens not to be modularized so there was never need for exports.
That's great. At least we can expect that fsnotify will be exported at some point and can start playing with fsnotify-based smb2_notify implementation.
Quoted 3 lines
Once NFS guys (or someone else) have something that is close to inclusion, I can push the export patch to my tree (just CC me, Amir, and fsdevel on the series).
OK, both OverlayFS and NFS have some plans for fsnotify, so let's see.
Quoted 3 lines
I prefer not to export symbols for something that is out of tree and will not be getting upstream in a near future. After all the export patch is easy to carry in your tree if you need it.
Makes perfect sense.
-ss