Skip to contentSkip to conversation

first draft of ZUFS - the Kernel part

RFC In review Boaz Harrosh 7 participants 39 messages Mar 13, 2018 – May 7, 2018 1 message not in the archive

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

cover first draft of ZUFS - the Kernel part

Hello all

I would please like to present the ZUFS file system and the Kernel code part in this patchset.

The Kernel code presented here can be found at:
https://github.com/NetApp/zufs-zuf

And for a very crud User-mode Server:
https://github.com/NetApp/zufs-zus

ZUFS - stands for Zero-copy User-mode FS
- It is geared towards true zero copy end to end of both data and meta data.
- It is geared towards very *low latency*, very high CPU locality, lock-less parallelism.
- Synchronous operations
- Numa awareness

Short description:
ZUFS is a from scratch implementation of a filesystem-in-user-space, which tries to address the above
goals. from the get go it is aimed for pmem based FSs. But can easily support other type of FSs that can utilize x10 latency and parallelism improvements. The novelty of this project is that the interface is designed with a modern multi-core NUMA machine in mind down to the ABI, so to reach these goals.

Not only FSs need apply, also any kind of user-mode Server can set up a pseudo filesystem and communicate with application via virtual files. These can then benefit from zero copy low-latency communication directly to/from application buffers. Or Application mmap direct Server resources. As long as it looks like a file system to the Kernel.

Current status is that we have couple trivial filesystem implementations and together with the Kernel module the UM-Server and the FSs User-mode pluggin can actually pass a good bunch of xfstests quick run. (Still working on Stability)

Just to get some points across as I said this project is all about performance and low latency. Here below are a POC results I have run:

In Kernel pmem-FS ZUFS FUSE
Threads Op/s Lat (us) Op/s Lat [us] Op/s Lat [us]
1 388361 2.271589 200799 4.6 71820 13.5
2 635115 2.604376 314321 5.9 148083 13.1
4 1260307 2.626361 565574 6.6 212133 18.3
8 2744963 2.485292 1113138 6.6 209799 37.6
12 2126945 5.020506 1598451 6.8 201689 58.7
18 4350995 3.386433 1648689 7.8 174823 101.8
24 4211180 4.784997 1702285 8 149413 159
36 3057166 9.291997 1783346 13.4 148276 240.7
48 3148972 10.382461 1741873 17.4 145296 327.3

I have used an average server machine in our lab with two NUMA nodes and total of 40 cores (Can't remember all the details). Running fio with 4k random writes. The IO is then just memcpy_nt() to a pmem simulated DRAM. The fio was run with more and more threads (see threads column)

We can see that we are still > x2 slower than in-Kernel FS. But I believe I can shave off another 1 us by optimizing the app-to-server thread switch by utilizing perhaps the "Binder" scheduler object or devising another way to not be going through the scheduler (and its locks) when switching VMs

BE-CAREFUL: This is a big code dump. And very much an RFC. Not yet very stable, not yet cleaned up. I have sliced the FS to 4 very big patches. Please talk to me if I should split it up to many more patches.

[I am afraid to send such huge emails so I'm posting web links instead. What is the mailing-list
message limit anyone knows?

For first version I'm sending web links to github of the 4 HUGE patches. Please clone above trees if you want to play with this.

Please tell me if you want to be removed from CC of these emails
]

list of patches:
[RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

This is a small but very important patch to the mmap code to support above scores. See inside the before and after results. (And is unfinished)

[RFC 2/7] fs: Add the ZUF filesystem to the build + License
Add the makefile and Kconfig + the licensing of the code

[RFC 3/7] zuf: Preliminary Documentation
Very unfinished Start of a Documentation for this project, Overall concept. Kernel side and usermode Server side.
Please help me in asking what you need more in here. Any questions I will try to add here

[RFC 4/7] zuf: zuf-rootfs && zuf-core
[RFC 5/7] zus: Devices && mounting
[RFC 6/7] zuf: Filesystem operations
[RFC 7/7] zuf: Write/Read && mmap implementation

After these 4 HUGE patches. There is a working live system. Still bugs and corner cases. But it can git clone and make a Kernel which is for me not so bad.

Thank you
Boaz

#1 lore raw

RFC 1/7 mm: Add new vma flag VM_LOCAL_CPU

On a call to mmap an mmap provider (like an FS) can put this flag on vma->vm_flags.

This tells the Kernel that the vma will be used from a single core only and therefore invalidation of PTE(s) need not a wide CPU scheduling

The motivation of this flag is the ZUFS project where we want to optimally map user-application buffers into a user-mode-server execute the operation and efficiently unmap.

In this project we utilize a per-core server thread so everything is kept local. If we use the regular zap_ptes() API All CPU's are scheduled for the unmap, though in our case we know that we have only used a single core. The regular zap_ptes adds a very big latency on every operations and mostly kills the concurrency of the over all system. Because it imposes a serialization between all cores

Some preliminary measurements on a 40 core machines:

unpatched patched
Threads Op/s Lat [us] Op/s Lat [us]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 185391 4.9 200799 4.6
2 197993 9.6 314321 5.9
4 310597 12.1 565574 6.6
8 546702 13.8 1113138 6.6
12 641728 17.2 1598451 6.8
18 744750 22.2 1648689 7.8
24 790805 28.3 1702285 8
36 849763 38.9 1783346 13.4
48 792000 44.6 1741873 17.4

[FIXME]
We need to actually impose this policy. On very first pte_insert we should sample the used CPU_ID and on all susequent pte_inserts we need to make sure it is the same CPU_ID used.

NOTE: That this vma is never used during a page_fault. It is always used in a synchronous way from an affinity set thread to a single core.

---
 fs/proc/task_mmu.c | 3 +++
 include/linux/mm.h | 3 +++
 mm/memory.c        | 2 +-
 3 files changed, 7 insertions(+), 1 deletion(-)
fs/proc/task_mmu.c +3 -0
index 339e4c1..20786ba 100644
@@ -681,6 +681,9 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma)
 		[ilog2(VM_PKEY_BIT2)]	= "",
 		[ilog2(VM_PKEY_BIT3)]	= "",
 #endif
+#ifdef CONFIG_ARCH_USES_HIGH_VMA_FLAGS
+		[ilog2(VM_LOCAL_CPU)]	= "lc",
+#endif
 	};
 	size_t i;
 
include/linux/mm.h +3 -0
index ea818ff..02bb8b5 100644
@@ -226,6 +226,9 @@ extern unsigned int kobjsize(const void *objp);
 #define VM_HIGH_ARCH_2	BIT(VM_HIGH_ARCH_BIT_2)
 #define VM_HIGH_ARCH_3	BIT(VM_HIGH_ARCH_BIT_3)
 #define VM_HIGH_ARCH_4	BIT(VM_HIGH_ARCH_BIT_4)
+#define VM_LOCAL_CPU	BIT(37)		/* FIXME: Needs to move from here */
+#else /* ! CONFIG_ARCH_USES_HIGH_VMA_FLAGS */
+#define VM_LOCAL_CPU	0		/* FIXME: Needs to move from here */
 #endif /* CONFIG_ARCH_USES_HIGH_VMA_FLAGS */
 
 #if defined(CONFIG_X86)
mm/memory.c +1 -1
index 7930046..7620ced 100644
@@ -1804,7 +1804,7 @@ static int insert_pfn(struct vm_area_struct *vma, unsigned long addr,
 				goto out_unlock;
 			entry = *pte;
 			goto out_mkwrite;
-		} else
+		} else if (!(vma->vm_flags & VM_LOCAL_CPU))
 			goto out_unlock;
 	}
 
Signature · 1 line
2.5.5
#2 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 10 lines · Boaz Harrosh

On a call to mmap an mmap provider (like an FS) can put this flag on vma->vm_flags.

This tells the Kernel that the vma will be used from a single core only and therefore invalidation of PTE(s) need not a wide CPU scheduling

The motivation of this flag is the ZUFS project where we want to optimally map user-application buffers into a user-mode-server execute the operation and efficiently unmap.

I've been looking at something similar, and I prefer my approach, although I'm not nearly as far along with my implementation as you are.

My approach is also to add a vm_flags bit, tentatively called VM_NOTLB. The page fault handler refuses to insert any TLB entries into the process address space. But follow_page_mask() will return the appropriate struct page for it. This should be enough for O_DIRECT accesses to work as you'll get the appropriate scatterlists built.

I suspect Boaz has already done a lot of thinking about this and doesn't need the explanation, but here's how it looks for anyone following along at home:

Process A calls read().
Kernel allocates a page cache page for it and calls the filesystem through
->readpages (or ->readpage).
Filesystem calls the managing process to get the data for that page. Managing process draws a pentagram and summons Beelzebub (or runs Perl;
whichever you find more scary).
Managing process notifies the filesystem that the page is now full of data. Filesystem marks the page as being Uptodate and unlocks it. Process was waiting on the page lock, wakes up and copies the data from the
page cache into userspace. read() is complete.

What we're concerned about here is what to do after the managing process tells the kernel that the read is complete. Clearly allowing the managing process continued access to the page is Bad as the page may be freed by the page cache and then reused for something else. Doing a TLB shootdown is expensive. So Boaz's approach is to have the process promise that it won't have any other thread look at it. My approach is to never allow the page to have load/store access from userspace; it can only be passed to other system calls.

#3 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 36 lines · Matthew Wilcox
Quoted 10 lines · Boaz Harrosh

On a call to mmap an mmap provider (like an FS) can put this flag on vma->vm_flags.

This tells the Kernel that the vma will be used from a single core only and therefore invalidation of PTE(s) need not a wide CPU scheduling

The motivation of this flag is the ZUFS project where we want to optimally map user-application buffers into a user-mode-server execute the operation and efficiently unmap.

I've been looking at something similar, and I prefer my approach, although I'm not nearly as far along with my implementation as you are.

My approach is also to add a vm_flags bit, tentatively called VM_NOTLB. The page fault handler refuses to insert any TLB entries into the process address space. But follow_page_mask() will return the appropriate struct page for it. This should be enough for O_DIRECT accesses to work as you'll get the appropriate scatterlists built.

I suspect Boaz has already done a lot of thinking about this and doesn't need the explanation, but here's how it looks for anyone following along at home:

Process A calls read().
Kernel allocates a page cache page for it and calls the filesystem through
->readpages (or ->readpage).
Filesystem calls the managing process to get the data for that page. Managing process draws a pentagram and summons Beelzebub (or runs Perl;
whichever you find more scary).
Managing process notifies the filesystem that the page is now full of data. Filesystem marks the page as being Uptodate and unlocks it. Process was waiting on the page lock, wakes up and copies the data from the
page cache into userspace. read() is complete.

Quoted 8 lines

What we're concerned about here is what to do after the managing process tells the kernel that the read is complete. Clearly allowing the managing process continued access to the page is Bad as the page may be freed by the page cache and then reused for something else. Doing a TLB shootdown is expensive. So Boaz's approach is to have the process promise that it won't have any other thread look at it. My approach is to never allow the page to have load/store access from userspace; it can only be passed to other system calls.

This all seems to revolve around the fact that userspace fs server process needs to copy something into userspace client's buffer, right?

Instead of playing with memory mappings, why not just tell the kernel *what* to copy?

While in theory not as generic, I don't see any real limitations (you don't actually need the current contents of the buffer in the read case and vica verse in the write case).

And we already have an interface for this: splice(2). What am I missing? What's the killer argument in favor of the above messing with tlb caches etc, instead of just letting the kernel do the dirty work.

Thanks,
Miklos

#4 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 52 lines · Miklos Szeredi
Quoted 36 lines · Matthew Wilcox
Quoted 10 lines · Boaz Harrosh

On a call to mmap an mmap provider (like an FS) can put this flag on vma->vm_flags.

This tells the Kernel that the vma will be used from a single core only and therefore invalidation of PTE(s) need not a wide CPU scheduling

The motivation of this flag is the ZUFS project where we want to optimally map user-application buffers into a user-mode-server execute the operation and efficiently unmap.

I've been looking at something similar, and I prefer my approach, although I'm not nearly as far along with my implementation as you are.

My approach is also to add a vm_flags bit, tentatively called VM_NOTLB. The page fault handler refuses to insert any TLB entries into the process address space. But follow_page_mask() will return the appropriate struct page for it. This should be enough for O_DIRECT accesses to work as you'll get the appropriate scatterlists built.

I suspect Boaz has already done a lot of thinking about this and doesn't need the explanation, but here's how it looks for anyone following along at home:

Process A calls read().
Kernel allocates a page cache page for it and calls the filesystem through
->readpages (or ->readpage).
Filesystem calls the managing process to get the data for that page. Managing process draws a pentagram and summons Beelzebub (or runs Perl;
whichever you find more scary).
Managing process notifies the filesystem that the page is now full of data. Filesystem marks the page as being Uptodate and unlocks it. Process was waiting on the page lock, wakes up and copies the data from the
page cache into userspace. read() is complete.

Quoted 8 lines

What we're concerned about here is what to do after the managing process tells the kernel that the read is complete. Clearly allowing the managing process continued access to the page is Bad as the page may be freed by the page cache and then reused for something else. Doing a TLB shootdown is expensive. So Boaz's approach is to have the process promise that it won't have any other thread look at it. My approach is to never allow the page to have load/store access from userspace; it can only be passed to other system calls.

This all seems to revolve around the fact that userspace fs server process needs to copy something into userspace client's buffer, right?

Instead of playing with memory mappings, why not just tell the kernel *what* to copy?

Quoted 8 lines

While in theory not as generic, I don't see any real limitations (you don't actually need the current contents of the buffer in the read case and vica verse in the write case).

And we already have an interface for this: splice(2). What am I missing? What's the killer argument in favor of the above messing with tlb caches etc, instead of just letting the kernel do the dirty work.

Great question. You're completely right that the question is how to tell the kernel what to copy. The problem is that splice() can only write to the first page of a pipe. So you need one pipe per outstanding request, which can easily turn into thousands of file descriptors. If we enhanced splice() so it could write to any page in a pipe, then I think splice() would be the perfect interface.

#5 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 62 lines · Matthew Wilcox
Quoted 52 lines · Miklos Szeredi
Quoted 36 lines · Matthew Wilcox
Quoted 10 lines · Boaz Harrosh

On a call to mmap an mmap provider (like an FS) can put this flag on vma->vm_flags.

This tells the Kernel that the vma will be used from a single core only and therefore invalidation of PTE(s) need not a wide CPU scheduling

The motivation of this flag is the ZUFS project where we want to optimally map user-application buffers into a user-mode-server execute the operation and efficiently unmap.

I've been looking at something similar, and I prefer my approach, although I'm not nearly as far along with my implementation as you are.

My approach is also to add a vm_flags bit, tentatively called VM_NOTLB. The page fault handler refuses to insert any TLB entries into the process address space. But follow_page_mask() will return the appropriate struct page for it. This should be enough for O_DIRECT accesses to work as you'll get the appropriate scatterlists built.

I suspect Boaz has already done a lot of thinking about this and doesn't need the explanation, but here's how it looks for anyone following along at home:

Process A calls read().
Kernel allocates a page cache page for it and calls the filesystem through
->readpages (or ->readpage).
Filesystem calls the managing process to get the data for that page. Managing process draws a pentagram and summons Beelzebub (or runs Perl;
whichever you find more scary).
Managing process notifies the filesystem that the page is now full of data. Filesystem marks the page as being Uptodate and unlocks it. Process was waiting on the page lock, wakes up and copies the data from the
page cache into userspace. read() is complete.

Quoted 8 lines

What we're concerned about here is what to do after the managing process tells the kernel that the read is complete. Clearly allowing the managing process continued access to the page is Bad as the page may be freed by the page cache and then reused for something else. Doing a TLB shootdown is expensive. So Boaz's approach is to have the process promise that it won't have any other thread look at it. My approach is to never allow the page to have load/store access from userspace; it can only be passed to other system calls.

This all seems to revolve around the fact that userspace fs server process needs to copy something into userspace client's buffer, right?

Instead of playing with memory mappings, why not just tell the kernel *what* to copy?

Quoted 8 lines

While in theory not as generic, I don't see any real limitations (you don't actually need the current contents of the buffer in the read case and vica verse in the write case).

And we already have an interface for this: splice(2). What am I missing? What's the killer argument in favor of the above messing with tlb caches etc, instead of just letting the kernel do the dirty work.

Quoted 6 lines

Great question. You're completely right that the question is how to tell the kernel what to copy. The problem is that splice() can only write to the first page of a pipe. So you need one pipe per outstanding request, which can easily turn into thousands of file descriptors. If we enhanced splice() so it could write to any page in a pipe, then I think splice() would be the perfect interface.

Don't know your usecase, but afaict zufs will have one queue per cpu. Having one pipe/cpu doesn't sound too bad.

But yeah, there's plenty of room for improvement in the splice interface. Just needs a killer app like this :)

Thanks,
Miklos

#6 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 13 lines · Miklos Szeredi
Quoted 11 lines · Matthew Wilcox
Quoted 4 lines

And we already have an interface for this: splice(2). What am I missing? What's the killer argument in favor of the above messing with tlb caches etc, instead of just letting the kernel do the dirty work.

Great question. You're completely right that the question is how to tell the kernel what to copy. The problem is that splice() can only write to the first page of a pipe. So you need one pipe per outstanding request, which can easily turn into thousands of file descriptors. If we enhanced splice() so it could write to any page in a pipe, then I think splice() would be the perfect interface.

Quoted 2 lines

Don't know your usecase, but afaict zufs will have one queue per cpu. Having one pipe/cpu doesn't sound too bad.

Erm ... there's nothing wrong with having one pipe per CPU. But pipes being non-seekable means that ZUFS can only handle synchronous I/Os. If you want to have a network backend, then you'd only be able to have one outstanding network request per pipe, which is really going to suck for bandwidth.

Quoted 2 lines

But yeah, there's plenty of room for improvement in the splice interface. Just needs a killer app like this :)

I'm happy to start investigating making pipes random-access ...

#7 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 14 lines · Matthew Wilcox
Quoted 13 lines · Miklos Szeredi
Quoted 11 lines · Matthew Wilcox
Quoted 4 lines

And we already have an interface for this: splice(2). What am I missing? What's the killer argument in favor of the above messing with tlb caches etc, instead of just letting the kernel do the dirty work.

Great question. You're completely right that the question is how to tell the kernel what to copy. The problem is that splice() can only write to the first page of a pipe. So you need one pipe per outstanding request, which can easily turn into thousands of file descriptors. If we enhanced splice() so it could write to any page in a pipe, then I think splice() would be the perfect interface.

Quoted 8 lines
Quoted 2 lines

Don't know your usecase, but afaict zufs will have one queue per cpu. Having one pipe/cpu doesn't sound too bad.

Erm ... there's nothing wrong with having one pipe per CPU. But pipes being non-seekable means that ZUFS can only handle synchronous I/Os. If you want to have a network backend, then you'd only be able to have one outstanding network request per pipe, which is really going to suck for bandwidth.

I guess ZUFS is mostly about fast synchronous access (please correct me if I'm wrong). Not sure that model fits network filesystems, where performance of caching will dominate real life performance.

Thanks,
Miklos

#8 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 10 lines · Miklos Szeredi
Quoted 5 lines · Matthew Wilcox

Erm ... there's nothing wrong with having one pipe per CPU. But pipes being non-seekable means that ZUFS can only handle synchronous I/Os. If you want to have a network backend, then you'd only be able to have one outstanding network request per pipe, which is really going to suck for bandwidth.

I guess ZUFS is mostly about fast synchronous access (please correct me if I'm wrong). Not sure that model fits network filesystems, where performance of caching will dominate real life performance.

I'm sure that's Boaz's use case ;-) But if we're introducing a replacement for FUSE, let's make it better than FUSE, not just specialised to Boaz's use case. Also, networks aren't necessarily slow; some of us live in a world where the other end-point on the "network" is *usually* the hypervisor, or a different guest on the same piece of physical hardware. Not to mention that 400Gbps ethernet is almost upon us (standard approved four months ago) and PCIe Gen 4 is only 256Gbps with a x16 link.

#9 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 8 lines · Matthew Wilcox
Quoted 7 lines · Miklos Szeredi
Quoted 5 lines · Matthew Wilcox

Erm ... there's nothing wrong with having one pipe per CPU. But pipes being non-seekable means that ZUFS can only handle synchronous I/Os. If you want to have a network backend, then you'd only be able to have one outstanding network request per pipe, which is really going to suck for bandwidth.

Quoted 7 lines
Quoted 3 lines

I guess ZUFS is mostly about fast synchronous access (please correct me if I'm wrong). Not sure that model fits network filesystems, where performance of caching will dominate real life performance.

I'm sure that's Boaz's use case ;-) But if we're introducing a replacement for FUSE, let's make it better than FUSE, not just specialised to Boaz's use case.

Okay, so the FUSE vs. ZUFS question was bound to be raised at some point. What's the high level thinking on this?

We can make ZUFS be a better FUSE, but with a new API? New API means
we'll lose existing user base. Having a new API but adding a compat
layer may be able to work around that, but it's probably hard to fully emulate the old API.

What are the limiting factors in the FUSE API that are preventing fixing these performance problems in FUSE?

Or it's just that FUSE kernel implementation is a horrid piece of bloat (true) and we need to do something from scratch without worrying about backward compat issues?

Is ZUFS planning to acquire a caching layer?

Btw, what is happening when a ZUFS file is mmaped?

Thanks,
Miklos

A message that is not in the archive (sent off-list or not yet received): CAON-v2ygEDCn90C9t-zadjsd5GRgj0ECqntQSDDtO_Zjk=KoVw@mail.gmail.com

#11 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 2 lines · Amit Golander

ZUFS is not intended or designed to replace FUSE. Just like RDMA is not intended to replace TCP.

Your intent is to have two independent approaches to permitting filesystems-in-userspace to exist. And I've pointed out that your approach doesn't work for my use-case (and neither does FUSE). While we might accept two implementations of something, at the point that three implementations are proposed, we usually say "No, come up with a better solution that works for everybody".

Quoted 44 lines · On Wed, 14 Mar 2018 at 8:39 Miklos Szeredi <[email protected]> wrote:
Quoted 43 lines
Quoted 2 lines · Matthew Wilcox
Quoted 1 line · Miklos Szeredi

On Wed, Mar 14, 2018 at 12:45 PM, Matthew Wilcox <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 4 lines

Erm ... there's nothing wrong with having one pipe per CPU. But pipes being non-seekable means that ZUFS can only handle synchronous I/Os. If you want to have a network backend, then you'd only be able to have one outstanding network request per pipe, which is really going to

suck

Quoted 9 lines
Quoted 5 lines
Quoted 1 line

for bandwidth.

I guess ZUFS is mostly about fast synchronous access (please correct me if I'm wrong). Not sure that model fits network filesystems, where performance of caching will dominate real life performance.

I'm sure that's Boaz's use case ;-) But if we're introducing a replacement for FUSE, let's make it better than FUSE, not just specialised to Boaz's use case.

Okay, so the FUSE vs. ZUFS question was bound to be raised at some point. What's the high level thinking on this?

We can make ZUFS be a better FUSE, but with a new API? New API means
we'll lose existing user base. Having a new API but adding a compat
layer may be able to work around that, but it's probably hard to fully emulate the old API.

What are the limiting factors in the FUSE API that are preventing fixing these performance problems in FUSE?

Or it's just that FUSE kernel implementation is a horrid piece of bloat (true) and we need to do something from scratch without worrying about backward compat issues?

Is ZUFS planning to acquire a caching layer?

Btw, what is happening when a ZUFS file is mmaped?

Thanks,
Miklos

#12 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

In reply to Miklos Szeredi
Quoted 37 lines · Miklos Szeredi
Quoted 36 lines · Matthew Wilcox
Quoted 10 lines · Boaz Harrosh

On a call to mmap an mmap provider (like an FS) can put this flag on vma->vm_flags.

This tells the Kernel that the vma will be used from a single core only and therefore invalidation of PTE(s) need not a wide CPU scheduling

The motivation of this flag is the ZUFS project where we want to optimally map user-application buffers into a user-mode-server execute the operation and efficiently unmap.

I've been looking at something similar, and I prefer my approach, although I'm not nearly as far along with my implementation as you are.

My approach is also to add a vm_flags bit, tentatively called VM_NOTLB. The page fault handler refuses to insert any TLB entries into the process address space. But follow_page_mask() will return the appropriate struct page for it. This should be enough for O_DIRECT accesses to work as you'll get the appropriate scatterlists built.

I suspect Boaz has already done a lot of thinking about this and doesn't need the explanation, but here's how it looks for anyone following along at home:

Process A calls read().
Kernel allocates a page cache page for it and calls the filesystem through
->readpages (or ->readpage).
Filesystem calls the managing process to get the data for that page. Managing process draws a pentagram and summons Beelzebub (or runs Perl;
whichever you find more scary).
Managing process notifies the filesystem that the page is now full of data. Filesystem marks the page as being Uptodate and unlocks it. Process was waiting on the page lock, wakes up and copies the data from the
page cache into userspace. read() is complete.

Quoted 9 lines
Quoted 8 lines

What we're concerned about here is what to do after the managing process tells the kernel that the read is complete. Clearly allowing the managing process continued access to the page is Bad as the page may be freed by the page cache and then reused for something else. Doing a TLB shootdown is expensive. So Boaz's approach is to have the process promise that it won't have any other thread look at it. My approach is to never allow the page to have load/store access from userspace; it can only be passed to other system calls.

Hi Matthew, Hi Miklos

Thank you for looking at this.
I'm answering both Matthew an Miklos's all thread, by trying to explain something that you might not have completely wrapped around yet.

Matthew first

Please note that in the ZUFS system there are no page-faults at all involved (God no, this is like +40us minimum and I'm fighting to shave off 13us)

In ZUF-to-ZUS communication
command comes in:
A1 we punch in the pages at the per-core-VMA before they are used, A2 we then return to user-space, access these pages once.
(without any page faults)
A3 Then return to kernel and punch in a drain page at that spot

New command comes in:
B1 we punch in the pages at the same per-core-VMA before they are used, B2 Return to user-space, access these new pages once.
B3 Then return to kernel and punch in a drain page at that spot

Actually I could skip A3/B3 all together but in testing after my patch it did not cost at all, so I like the extra easiness (Because otherwise there is a dance I need to do when app or server crash and files start to close I need to scan VMAs and zap them)

Current mm's mapping code (at insert_pfn) will fail at B1 above. Because it wants to see a ZERO empty spot before inserting a new pte. What the mm code wants is that I call

	A3 - zap_vma_ptes(vma)

This is because if the spot was not ZERO it means there was a previous mapping there. And some other core might have cached that entry at the TLB. so when I punch in this new value the other core could access the old page while this core is accessing the new page. (TLB-invalidate is a single core command and is why zap_vma_ptes
needs to schedule all cores to each call TLB-invalidate)

And this is the all difference between the two testes above. That I do not zap_vma_ptes With the new (one liner) code.

Please Note that the VM_LOCAL_CPU flag is not set by the application (zus Server) but by the Kernel driver, telling the Kernel that it has enforced such an API that we access from a single CORE so please allow me B1 because I know what I'm doing. (Also we do put some trust into zus because it has our filesystem data and because we wrote it ;-))

I understand your approach where you say "The PTE table is just a global communicator of pages but is not really mapped into any process .i.e never faulted into any core's local-TLB" (The Kernel access of that memory is done on a Kernel address at another TLB). And is why I can get away from zap_vma_ptes(vma).
So is this not the same thing? your flag says no one TLB cached this PTE my flag says only-this-core-cached this PTE. We both ask
"So please skip the zap_vma_ptes(vma) stage for me"

I think you might be able to use my flag for your system. Is only a small part of what you need with the all "Get the page from the PTE at" and so on. But the "please skip zap_vma_ptes(vma)" part is this patch here, No?

BTW I did not at all understand what is your project trying to solve. please send me some Notes about it I want to see if they might fit after all

Quoted 10 lines

This all seems to revolve around the fact that userspace fs server process needs to copy something into userspace client's buffer, right?

Instead of playing with memory mappings, why not just tell the kernel *what* to copy?

While in theory not as generic, I don't see any real limitations (you don't actually need the current contents of the buffer in the read case and vica verse in the write case).

This is not so easy, for many reasons. It was actually my first approach which I pursued for a while but dropped it for the easier to implement and more general approach.

Note that we actually do that in the implementation of mmap. There is a ZUS_OP_GET_BLOCK which returns a dpp_t of a page to map into the application's VM. We could just copy it at that point

We have some app buffers arriving with pointers local to one VM (the app) and then we want to copy them to another app buffers. How do you do that? So yes you need to get_user_pages() so they can be accessed from kernel, switch to second VM then receive pointers there. These need to be dpp_t like the games I do, or - In the app context copy_user_to_page.

But that API was not enough for me. Because this is good with pmem. But what if I actually want it from disk or network. My API you can do that easily without any copy or caching still.
Not in this RFC - but there is a plan (Is my very next todo) for an ASYNC operation mode as well as the sync operation. The zus is telling ZUF => -ASYNC please the data you wanted is on slow media I need to sleep. The request is put on hold and completed in the background. An async thread will later call to complete the command. Note that in that case we will do zap_vma_ptes(vma). And back to square one. But in that case the cost of zap_vma_ptes(vma) is surly accepted.

Also there was a very big locking problem with the OP_GET_BLOCK approach. Because the while a copy is made, FS needs to lock access to that same page in many kind of scenarios. Just few examples:
1- COW writer a concurrent reader should see the old data.
2- unwritten-buffer-write - concurrent reader should see zeros which
means I need to write zeros first, before letting reads in. Grrr this is current DAX code. I know how to do better
3- tier-down - I want to write a page to slow media and reuse it. Must not
allow this while it is accessed.

And many more. So in all these cases the API will need to be OP_GET_BLOCK / OP_PUT_BLOCK which is two trips. Fffff very slow.

And specially in the network or from-device case, the zus server needs to now have all these buffer cache management and life time hell. because it needs to read this data somewhere before it presents the page back to Kernel, and there you have a COPY for you.

In my API you can network directly to the APP buffers they are there why not use them. (Did I say ZERO copy ;-) )

Also Psuedo FS application servers say like MySQL-5. OP_GET_BLOCK will give it a big memory management problem. where now we can just write directly to app buffers, and again in zero copy.

Please note that it will be very easy with this API to also support page-cache for FSs that wants it like the network FSs you said.
The FS will set a bit in the fs_register call to say that it would rather use page cache. These type of FSs will run on a different
kind of BDI which says "Yes page cache please". All the IO entry vectors point to the generic_iter API and instead we implement read/write_pages(). At read/write_pages() we do the exact same OP_READ/WRITE like today. map the cache pages to the zus VM, despatch, return, release page_lock. all is happy. Any one wanting to contribute this is very welcome.

I did have plans in that first approach to have a cache of OP_GET_BLOCKs on the radix tree. And have the Server recall these blocks when needed. But this called for alot of locking on the HOT path. And was much much more complicated, bigger code.
Here we have a completely lockless, zero synchronization between cores code. With the one liner of this patch even the all vma_mapping is lockless And is so very simple, with a huge gain and no loss. Because ....

You said above: "Instead of playing with memory mappings"

But if you look at the amount of code, even compared to a pipe or spline. You will see that the "playing with memory mappings" is so very easy and simple. It might be new, hard to grasp approach but it is just harder as a new concept then an actually code complexity. All I actually do is:

1. Allocate a vma per core
2. call vm_insert_pfn
.... Do something
3. vm_insert_pfn(NULL) (before this patch zap_vma_ptes())

It is all very simple really. For me it is opposite. It is "Why mess around with dual_port_pointers, caching, and copy
life time rules, when you can just call vm_insert_pfn"

Quoted 5 lines

And we already have an interface for this: splice(2). What am I missing? What's the killer argument in favor of the above messing with tlb caches etc, instead of just letting the kernel do the dirty work.

You answered yourself. We are the Kernel and we are doing the (simple) work. If you look at all this from far, the zus-core with its Z-Threads array is just a fancy pipe really - A zero copy pipe.

Being a splice API gives us nothing. It will have the same problems as above. splice basically says Party A show me your buffers Party B show me yours, and I can copy between them in the Kernel. Usually one of them A or B is in Kernel buffers or a DMA target. So this case is very like the OPT_GET_BLOCK you have life time problems. And if you use the direct mmaped pipe like you talked to Matthew about then you are back to this exact problem and with current API you cannot avoid neither the zap_vma_ptes() nor actual page-faults after the mmap. So you are looking at 60u minimum I have the all round trip in 4.6u. And I believe I can cut it down to 3.5u with fixing that Relay object

I have researched this for a while. I do not believe there is a more rubust, and certainly this one liner is not complexity either.

Quoted 3 lines

Thanks,
Miklos

I Hope this sheds some light on the matter.

Thankyou
Boaz

#13 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 133 lines · Boaz Harrosh
Quoted 37 lines · Miklos Szeredi
Quoted 36 lines · Matthew Wilcox
Quoted 10 lines · Boaz Harrosh

On a call to mmap an mmap provider (like an FS) can put this flag on vma->vm_flags.

This tells the Kernel that the vma will be used from a single core only and therefore invalidation of PTE(s) need not a wide CPU scheduling

The motivation of this flag is the ZUFS project where we want to optimally map user-application buffers into a user-mode-server execute the operation and efficiently unmap.

I've been looking at something similar, and I prefer my approach, although I'm not nearly as far along with my implementation as you are.

My approach is also to add a vm_flags bit, tentatively called VM_NOTLB. The page fault handler refuses to insert any TLB entries into the process address space. But follow_page_mask() will return the appropriate struct page for it. This should be enough for O_DIRECT accesses to work as you'll get the appropriate scatterlists built.

I suspect Boaz has already done a lot of thinking about this and doesn't need the explanation, but here's how it looks for anyone following along at home:

Process A calls read().
Kernel allocates a page cache page for it and calls the filesystem through
->readpages (or ->readpage).
Filesystem calls the managing process to get the data for that page. Managing process draws a pentagram and summons Beelzebub (or runs Perl;
whichever you find more scary).
Managing process notifies the filesystem that the page is now full of data. Filesystem marks the page as being Uptodate and unlocks it. Process was waiting on the page lock, wakes up and copies the data from the
page cache into userspace. read() is complete.

Quoted 9 lines
Quoted 8 lines

What we're concerned about here is what to do after the managing process tells the kernel that the read is complete. Clearly allowing the managing process continued access to the page is Bad as the page may be freed by the page cache and then reused for something else. Doing a TLB shootdown is expensive. So Boaz's approach is to have the process promise that it won't have any other thread look at it. My approach is to never allow the page to have load/store access from userspace; it can only be passed to other system calls.

Hi Matthew, Hi Miklos

Thank you for looking at this.
I'm answering both Matthew an Miklos's all thread, by trying to explain something that you might not have completely wrapped around yet.

Matthew first

Please note that in the ZUFS system there are no page-faults at all involved (God no, this is like +40us minimum and I'm fighting to shave off 13us)

In ZUF-to-ZUS communication
command comes in:
A1 we punch in the pages at the per-core-VMA before they are used, A2 we then return to user-space, access these pages once.
(without any page faults)
A3 Then return to kernel and punch in a drain page at that spot

New command comes in:
B1 we punch in the pages at the same per-core-VMA before they are used, B2 Return to user-space, access these new pages once.
B3 Then return to kernel and punch in a drain page at that spot

Actually I could skip A3/B3 all together but in testing after my patch it did not cost at all, so I like the extra easiness (Because otherwise there is a dance I need to do when app or server crash and files start to close I need to scan VMAs and zap them)

Current mm's mapping code (at insert_pfn) will fail at B1 above. Because it wants to see a ZERO empty spot before inserting a new pte. What the mm code wants is that I call

        A3 - zap_vma_ptes(vma)

This is because if the spot was not ZERO it means there was a previous mapping there. And some other core might have cached that entry at the TLB. so when I punch in this new value the other core could access the old page while this core is accessing the new page. (TLB-invalidate is a single core command and is why zap_vma_ptes
needs to schedule all cores to each call TLB-invalidate)

And this is the all difference between the two testes above. That I do not zap_vma_ptes With the new (one liner) code.

Please Note that the VM_LOCAL_CPU flag is not set by the application (zus Server) but by the Kernel driver, telling the Kernel that it has enforced such an API that we access from a single CORE so please allow me B1 because I know what I'm doing. (Also we do put some trust into zus because it has our filesystem data and because we wrote it ;-))

I understand your approach where you say "The PTE table is just a global communicator of pages but is not really mapped into any process .i.e never faulted into any core's local-TLB" (The Kernel access of that memory is done on a Kernel address at another TLB). And is why I can get away from zap_vma_ptes(vma).
So is this not the same thing? your flag says no one TLB cached this PTE my flag says only-this-core-cached this PTE. We both ask
"So please skip the zap_vma_ptes(vma) stage for me"

I think you might be able to use my flag for your system. Is only a small part of what you need with the all "Get the page from the PTE at" and so on. But the "please skip zap_vma_ptes(vma)" part is this patch here, No?

BTW I did not at all understand what is your project trying to solve. please send me some Notes about it I want to see if they might fit after all

Quoted 10 lines

This all seems to revolve around the fact that userspace fs server process needs to copy something into userspace client's buffer, right?

Instead of playing with memory mappings, why not just tell the kernel *what* to copy?

While in theory not as generic, I don't see any real limitations (you don't actually need the current contents of the buffer in the read case and vica verse in the write case).

This is not so easy, for many reasons. It was actually my first approach which I pursued for a while but dropped it for the easier to implement and more general approach.

Note that we actually do that in the implementation of mmap. There is a ZUS_OP_GET_BLOCK which returns a dpp_t of a page to map into the application's VM. We could just copy it at that point

Quoted 8 lines

We have some app buffers arriving with pointers local to one VM (the app) and then we want to copy them to another app buffers. How do you do that? So yes you need to get_user_pages() so they can be accessed from kernel, switch to second VM then receive pointers there. These need to be dpp_t like the games I do, or - In the app context copy_user_to_page.

But that API was not enough for me. Because this is good with pmem. But what if I actually want it from disk or network.

Yeah, that's the interesting part. We want direct-io into the client (the app) memory, with the userspace filesystem acting as a traffic controller.

With your scheme it's like:

- get_user_pages
- map pages into server address space
- send request to server
- server does direct-io read from network/disk fd into mapped buffer
- server sends reply
- done

This could be changed to
- get_user_pages
- insert pages into pipe
- send request to server
- server "reverse splices" buffers from pipe to network/disk fd
- server sends reply
- done

The two are basically the same, except we got rid of the unnecessary userspace mapping.

Okay, the "reverse splice" or "rsplice" operation is yet to be defined. It would be like splice, except it passes an empty buffer from the pipe into an operation that uses it to fill the buffer (RSPLICE is to SPLICE as READ is to WRITE).

For write operation the normal splice(2) would be used in the same way, straightforward passing of user buffer directly to underlying device without memory copy ever being done.

See what I'm getting at?

Quoted 8 lines

1. Allocate a vma per core
2. call vm_insert_pfn
.... Do something
3. vm_insert_pfn(NULL) (before this patch zap_vma_ptes())

It is all very simple really. For me it is opposite. It is "Why mess around with dual_port_pointers, caching, and copy
life time rules, when you can just call vm_insert_pfn"

Because you normally gain nothing by going through the server address space.

Mapping to server address space has issues, like allowing access from server to full page containing the buffer, which might well be a security issue.

Also with the direct-io from network/disk case the userspace address will again be translated to a page in the kernel so it's just going back and forth between representations using the page tables, which likely even results in a measurable performance loss.

Again, what's the advantage of mapping to server address space?

Thanks,
Miklos

#14 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 140 lines · Miklos Szeredi
Quoted 139 lines · Boaz Harrosh
Quoted 37 lines · Miklos Szeredi
Quoted 36 lines · Matthew Wilcox
Quoted 10 lines · Boaz Harrosh

On a call to mmap an mmap provider (like an FS) can put this flag on vma->vm_flags.

This tells the Kernel that the vma will be used from a single core only and therefore invalidation of PTE(s) need not a wide CPU scheduling

The motivation of this flag is the ZUFS project where we want to optimally map user-application buffers into a user-mode-server execute the operation and efficiently unmap.

I've been looking at something similar, and I prefer my approach, although I'm not nearly as far along with my implementation as you are.

My approach is also to add a vm_flags bit, tentatively called VM_NOTLB. The page fault handler refuses to insert any TLB entries into the process address space. But follow_page_mask() will return the appropriate struct page for it. This should be enough for O_DIRECT accesses to work as you'll get the appropriate scatterlists built.

I suspect Boaz has already done a lot of thinking about this and doesn't need the explanation, but here's how it looks for anyone following along at home:

Process A calls read().
Kernel allocates a page cache page for it and calls the filesystem through
->readpages (or ->readpage).
Filesystem calls the managing process to get the data for that page. Managing process draws a pentagram and summons Beelzebub (or runs Perl;
whichever you find more scary).
Managing process notifies the filesystem that the page is now full of data. Filesystem marks the page as being Uptodate and unlocks it. Process was waiting on the page lock, wakes up and copies the data from the
page cache into userspace. read() is complete.

Quoted 9 lines
Quoted 8 lines

What we're concerned about here is what to do after the managing process tells the kernel that the read is complete. Clearly allowing the managing process continued access to the page is Bad as the page may be freed by the page cache and then reused for something else. Doing a TLB shootdown is expensive. So Boaz's approach is to have the process promise that it won't have any other thread look at it. My approach is to never allow the page to have load/store access from userspace; it can only be passed to other system calls.

Hi Matthew, Hi Miklos

Thank you for looking at this.
I'm answering both Matthew an Miklos's all thread, by trying to explain something that you might not have completely wrapped around yet.

Matthew first

Please note that in the ZUFS system there are no page-faults at all involved (God no, this is like +40us minimum and I'm fighting to shave off 13us)

In ZUF-to-ZUS communication
command comes in:
A1 we punch in the pages at the per-core-VMA before they are used, A2 we then return to user-space, access these pages once.
(without any page faults)
A3 Then return to kernel and punch in a drain page at that spot

New command comes in:
B1 we punch in the pages at the same per-core-VMA before they are used, B2 Return to user-space, access these new pages once.
B3 Then return to kernel and punch in a drain page at that spot

Actually I could skip A3/B3 all together but in testing after my patch it did not cost at all, so I like the extra easiness (Because otherwise there is a dance I need to do when app or server crash and files start to close I need to scan VMAs and zap them)

Current mm's mapping code (at insert_pfn) will fail at B1 above. Because it wants to see a ZERO empty spot before inserting a new pte. What the mm code wants is that I call

        A3 - zap_vma_ptes(vma)

This is because if the spot was not ZERO it means there was a previous mapping there. And some other core might have cached that entry at the TLB. so when I punch in this new value the other core could access the old page while this core is accessing the new page. (TLB-invalidate is a single core command and is why zap_vma_ptes
needs to schedule all cores to each call TLB-invalidate)

And this is the all difference between the two testes above. That I do not zap_vma_ptes With the new (one liner) code.

Please Note that the VM_LOCAL_CPU flag is not set by the application (zus Server) but by the Kernel driver, telling the Kernel that it has enforced such an API that we access from a single CORE so please allow me B1 because I know what I'm doing. (Also we do put some trust into zus because it has our filesystem data and because we wrote it ;-))

I understand your approach where you say "The PTE table is just a global communicator of pages but is not really mapped into any process .i.e never faulted into any core's local-TLB" (The Kernel access of that memory is done on a Kernel address at another TLB). And is why I can get away from zap_vma_ptes(vma).
So is this not the same thing? your flag says no one TLB cached this PTE my flag says only-this-core-cached this PTE. We both ask
"So please skip the zap_vma_ptes(vma) stage for me"

I think you might be able to use my flag for your system. Is only a small part of what you need with the all "Get the page from the PTE at" and so on. But the "please skip zap_vma_ptes(vma)" part is this patch here, No?

BTW I did not at all understand what is your project trying to solve. please send me some Notes about it I want to see if they might fit after all

Quoted 10 lines

This all seems to revolve around the fact that userspace fs server process needs to copy something into userspace client's buffer, right?

Instead of playing with memory mappings, why not just tell the kernel *what* to copy?

While in theory not as generic, I don't see any real limitations (you don't actually need the current contents of the buffer in the read case and vica verse in the write case).

This is not so easy, for many reasons. It was actually my first approach which I pursued for a while but dropped it for the easier to implement and more general approach.

Note that we actually do that in the implementation of mmap. There is a ZUS_OP_GET_BLOCK which returns a dpp_t of a page to map into the application's VM. We could just copy it at that point

We have some app buffers arriving with pointers local to one VM (the app) and then we want to copy them to another app buffers. How do you do that? So yes you need to get_user_pages() so they can be accessed from kernel, switch to second VM then receive pointers there. These need to be dpp_t like the games I do, or - In the app context copy_user_to_page.

Quoted 7 lines
Quoted 2 lines

But that API was not enough for me. Because this is good with pmem. But what if I actually want it from disk or network.

Yeah, that's the interesting part. We want direct-io into the client (the app) memory, with the userspace filesystem acting as a traffic controller.

(Yes we want that, but is not the only thing we want. We also want
pmem, and other Server available data)

Quoted 9 lines

With your scheme it's like:

- get_user_pages
- map pages into server address space
- send request to server
- server does direct-io read from network/disk fd into mapped buffer
- server sends reply
- done

Quoted 5 lines

This could be changed to
- get_user_pages
- insert pages into pipe
- send request to server
- server "reverse splices" buffers from pipe to network/disk fd

This can never properly translate. Even a simple file on disk is linear for the app (unaligned buffer) but is scattered on multiple blocks on disk. Yes perhaps networking can somewhat work if you pre/post-pend the headers you need.
And you restrict direct IO semantics on everything specially the APP with my system you can do zero copy on any kind of application

And this assumes networking or some-device. Which means going back to the Kernel, which in ZUFS rules you must return -ASYNC to the zuf and complete in a background ASYNC thread. This is an order of a magnitude higher latency then what I showed here.

And what about the SYNC copy from Server to APP. With a pipe you are forcing me to go back to the Kernel to execute the copy. which means two more crossings. This will double the round trips.

Quoted 11 lines

- server sends reply
- done

The two are basically the same, except we got rid of the unnecessary userspace mapping.

Okay, the "reverse splice" or "rsplice" operation is yet to be defined. It would be like splice, except it passes an empty buffer from the pipe into an operation that uses it to fill the buffer (RSPLICE is to SPLICE as READ is to WRITE).

Exactly another trip back to Kernel.

Quoted 4 lines

For write operation the normal splice(2) would be used in the same way, straightforward passing of user buffer directly to underlying device without memory copy ever being done.

Here too, can work for direct access semantics with pointers and sizes aligned, but cannot satisfy POSIX.

Quoted 2 lines

See what I'm getting at?

Do you see my points?
- Zero-copy on all posix API, including *no* page-cache.
- A single kernel-UM transition.
- Synchronous extreme low latency.

Quoted 9 lines
Quoted 8 lines

1. Allocate a vma per core
2. call vm_insert_pfn
.... Do something
3. vm_insert_pfn(NULL) (before this patch zap_vma_ptes())

It is all very simple really. For me it is opposite. It is "Why mess around with dual_port_pointers, caching, and copy
life time rules, when you can just call vm_insert_pfn"

Quoted 6 lines

Because you normally gain nothing by going through the server address space.

Mapping to server address space has issues, like allowing access from server to full page containing the buffer, which might well be a security issue.

Not really there is already an high trust between the APP and the filesystem Server owning the all of the APP's data. A compromised Server can do lots and lots of bad things before a bug trashes the unaligned tails of a buffer.
(And at that the Server only has access to IO buffers in the short window
of the IO execution. Once on IO return this access is disconnected)

Quoted 5 lines

Also with the direct-io from network/disk case the userspace address will again be translated to a page in the kernel so it's just going back and forth between representations using the page tables, which likely even results in a measurable performance loss.

You mean another get_user_pages() This is not so bad we have all these pages already paged in and the page-table HOT because we just now set it. From my tests I did not notice any such slowness.

Again this is not my typical work load, and this extra get_user_pages() is not an high priority for me. But if you really care about it and you measure real slowness because of that extra get_user_pages() what we can do is:
I already have a zuf internal object describing the request
including the app mapped pages and sizes. We can implement a splice() operation on the zuf driver. As a target. To supply the array of pages already gotten at the first above get_user_pages().

Quoted 2 lines

Again, what's the advantage of mapping to server address space?

See above. In my case 90% of the time the data is already at the Server application memcpy_nt away. If I want to support that mode of a single trip to user-land. That is the way.

If you are positive you have 2 trips minimum and the target is in kernel. You are already too slow and there is not much of a difference.

So the advantage is the extra choice. Which for me is the 90% of the work load.

Quoted 3 lines

Thanks,
Miklos

Thanks
Boaz

#15 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 6 lines · Boaz Harrosh

Not really there is already an high trust between the APP and the filesystem Server owning the all of the APP's data. A compromised Server can do lots and lots of bad things before a bug trashes the unaligned tails of a buffer.
(And at that the Server only has access to IO buffers in the short window
of the IO execution. Once on IO return this access is disconnected)

Without a TLB shootdown, you can't guarantee that. Here's how it works:

CPU A is notified of a new page, starts accessing the page.
CPU B decides to access the same page
CPU A notifies the kernel
Kernel withdraws the PTE mapping, but doesn't zap it.
CPU B can still access the page until whatever CPU magic happens to discard the PTE from the TLB.
Kernel decides to recycle the page
Kernel allocates it to some kernel data structure
CPU B writes to it, can probably escalate to kernel privileges.

Now, you're going to argue that the process is trusted and should be considered to be part of the kernel from a trust point of view. In that case it needs to be distributed as part of the kernel and not be an independent user process.

#16 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 20 lines · Matthew Wilcox
Quoted 6 lines · Boaz Harrosh

Not really there is already an high trust between the APP and the filesystem Server owning the all of the APP's data. A compromised Server can do lots and lots of bad things before a bug trashes the unaligned tails of a buffer.
(And at that the Server only has access to IO buffers in the short window
of the IO execution. Once on IO return this access is disconnected)

Without a TLB shootdown, you can't guarantee that. Here's how it works:

CPU A is notified of a new page, starts accessing the page.
CPU B decides to access the same page
CPU A notifies the kernel
Kernel withdraws the PTE mapping, but doesn't zap it.
CPU B can still access the page until whatever CPU magic happens to discard the PTE from the TLB.
Kernel decides to recycle the page
Kernel allocates it to some kernel data structure
CPU B writes to it, can probably escalate to kernel privileges.

Quoted 5 lines

Now, you're going to argue that the process is trusted and should be considered to be part of the kernel from a trust point of view. In that case it needs to be distributed as part of the kernel and not be an independent user process.

You are right in the General case but this is not the case for ZUF.

The buffers belong to the application, So it is all about the zus Server having access to the APP buffers but please look exactly what zufs does:
(Repeated from above)

On the same exact core always (zufs rules ZTs and all)

Quoted 2 lines

A1 we punch in the pages at the per-core-VMA before they are used, A2 we then return to user-space, access these pages once.

from this core only

Quoted 1 line

A3 Then return to kernel and punch in a drain page at that spot

At A1 A3 stages there is a local TLB-invalidate for the single core but not a system-wide one. So the window of where the server has access to app buffers is during the IO and not passed it.

There is a trust that zus server will not access the per-core private vma from another core, yes. But the way it is implemented this is very hard to do. Because the vma sits on a O_TMPFILE+exclusive private to the affinity set thread's stack and is not public to anyone but this single Z-Thread.

Yes if the zus will access that vma from another core what you say, will happen but this can only happen on compromised server. And a compromised Server can mess up the applications IO buffers through the front door so why worry about a back door. Perhaps I'm missing something?

Thanks
Boaz

#17 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

In reply to Boaz Harrosh
Quoted 17 lines · Boaz Harrosh
Quoted 9 lines · Miklos Szeredi

With your scheme it's like:

- get_user_pages
- map pages into server address space
- send request to server
- server does direct-io read from network/disk fd into mapped buffer
- server sends reply
- done

Quoted 5 lines

This could be changed to
- get_user_pages
- insert pages into pipe
- send request to server
- server "reverse splices" buffers from pipe to network/disk fd

Quoted 6 lines

This can never properly translate. Even a simple file on disk is linear for the app (unaligned buffer) but is scattered on multiple blocks on disk. Yes perhaps networking can somewhat work if you pre/post-pend the headers you need.
And you restrict direct IO semantics on everything specially the APP with my system you can do zero copy on any kind of application

I lost you there, sorry.

How will your scheme deal with alignment issues better than my scheme?

Quoted 4 lines

And this assumes networking or some-device. Which means going back to the Kernel, which in ZUFS rules you must return -ASYNC to the zuf and complete in a background ASYNC thread. This is an order of a magnitude higher latency then what I showed here.

Indeed.

Quoted 3 lines

And what about the SYNC copy from Server to APP. With a pipe you are forcing me to go back to the Kernel to execute the copy. which means two more crossings. This will double the round trips.

If you are trying to minimize the roundtrips, why not cache the mapping in the kernel? That way you don't necessarily have to go to userspace at all. With readahead logic, the server will be able to preload the mapping before the reads happen, and you basically get the same speed as an in-kernel fs would.

Also don't quite understand how are you planning to generalize beyond the pmem case. The interface is ready for that, sure. But what about
caching? Will that be done in the server? Does that make sense?
Kernel already has page cache for that purpose and userspace cache won't ever be as good as kernel cache.

Thanks,
Miklos

#18 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

On 15/03/18 18:10, Miklos Szeredi wrote:
<>

Quoted 11 lines
Quoted 6 lines

This can never properly translate. Even a simple file on disk is linear for the app (unaligned buffer) but is scattered on multiple blocks on disk. Yes perhaps networking can somewhat work if you pre/post-pend the headers you need.
And you restrict direct IO semantics on everything specially the APP with my system you can do zero copy on any kind of application

I lost you there, sorry.

How will your scheme deal with alignment issues better than my scheme?

In my pmem case easy memcpy. This will not work if you need to go to hard disk I agree. (Which is not a priority for me)

Quoted 11 lines
Quoted 4 lines

And this assumes networking or some-device. Which means going back to the Kernel, which in ZUFS rules you must return -ASYNC to the zuf and complete in a background ASYNC thread. This is an order of a magnitude higher latency then what I showed here.

Indeed.

Quoted 3 lines

And what about the SYNC copy from Server to APP. With a pipe you are forcing me to go back to the Kernel to execute the copy. which means two more crossings. This will double the round trips.

Quoted 6 lines

If you are trying to minimize the roundtrips, why not cache the mapping in the kernel? That way you don't necessarily have to go to userspace at all. With readahead logic, the server will be able to preload the mapping before the reads happen, and you basically get the same speed as an in-kernel fs would.

Yes as I said that was my first approach. But at the end this is always a special workload optimization but in the general case this actually adds a round trip and a huge complexity that always comes to bite you.

Quoted 6 lines

Also don't quite understand how are you planning to generalize beyond the pmem case. The interface is ready for that, sure. But what about
caching? Will that be done in the server? Does that make sense?
Kernel already has page cache for that purpose and userspace cache won't ever be as good as kernel cache.

I explained about that. We can easily support page-cache in zufs

Quoted 9 lines · here what I

Please note that it will be very easy with this API to also support page-cache for FSs that wants it like the network FSs you said.
The FS will set a bit in the fs_register call to say that it would rather use page cache. These type of FSs will run on a different
kind of BDI which says "Yes page cache please". All the IO entry vectors point to the generic_iter API and instead we implement read/write_pages(). At read/write_pages() we do the exact same OP_READ/WRITE like today. map the cache pages to the zus VM, despatch, return, release page_lock. all is happy. Any one wanting to contribute this is very welcome.

Yes please no caching at the zus level that's insane

Quoted 3 lines

Thanks,
Miklos

Thanks
Boaz

#19 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 35 lines · Boaz Harrosh

On 15/03/18 18:10, Miklos Szeredi wrote:
<>

Quoted 11 lines
Quoted 6 lines

This can never properly translate. Even a simple file on disk is linear for the app (unaligned buffer) but is scattered on multiple blocks on disk. Yes perhaps networking can somewhat work if you pre/post-pend the headers you need.
And you restrict direct IO semantics on everything specially the APP with my system you can do zero copy on any kind of application

I lost you there, sorry.

How will your scheme deal with alignment issues better than my scheme?

In my pmem case easy memcpy. This will not work if you need to go to hard disk I agree. (Which is not a priority for me)

Quoted 11 lines
Quoted 4 lines

And this assumes networking or some-device. Which means going back to the Kernel, which in ZUFS rules you must return -ASYNC to the zuf and complete in a background ASYNC thread. This is an order of a magnitude higher latency then what I showed here.

Indeed.

Quoted 3 lines

And what about the SYNC copy from Server to APP. With a pipe you are forcing me to go back to the Kernel to execute the copy. which means two more crossings. This will double the round trips.

Quoted 6 lines

If you are trying to minimize the roundtrips, why not cache the mapping in the kernel? That way you don't necessarily have to go to userspace at all. With readahead logic, the server will be able to preload the mapping before the reads happen, and you basically get the same speed as an in-kernel fs would.

Quoted 4 lines

Yes as I said that was my first approach. But at the end this is always a special workload optimization but in the general case this actually adds a round trip and a huge complexity that always comes to bite you.

Ideally most of the complexity would be in the page cache. Not sure how ready it is to handle pmem pages?

The general case (non-pmem) will always have to be handled differently; you've just stated that it's much less latency sensitive
and needs async handling. Basing the design on just trying to make
it use the same mechanism (userspace copy) is flawed in my opinion, since it's suboptimal for either case.

Thanks,
Miklos

#20 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 11 lines · Miklos Szeredi

Ideally most of the complexity would be in the page cache. Not sure how ready it is to handle pmem pages?

The general case (non-pmem) will always have to be handled differently; you've just stated that it's much less latency sensitive
and needs async handling. Basing the design on just trying to make
it use the same mechanism (userspace copy) is flawed in my opinion, since it's suboptimal for either case.

Thanks,
Miklos

OK So I was thinking hard on all this and am changing my mind and agreeing with all that was said.

I want that the usFS plugin will have all the different options and have an easy way to tell Kernel which mode to use.

Let me summarize all the options:

1. Sync, userspace copy directly to app-buffers (current implementation)

2. Async block device operation (none pmem)
    zuf owns all devices pmem and none pmem at mount time and provides
    a very efficient access to both. In the harddisk / ssd case as part of an IO call the server returns -EWOULD_BLOCK and in the background will issue a scatter_gather call through zuf.
    The memory target for the IO can be pmem, directly to user-buffers (DIO), transient server buffers.
     On completion an up call is made to ZUF to complete the IO operation and release the waiting application.

3. Splice and R-spilce
    In the case that the IO target is not a block-device but an external path like network / rdma / some none block device.
    Zuf already holds an internal object describing the IO context including the GUP app buffers. This internal object can be made the memory target of a splice operation.

4. Get-io_map type operation (currently implemented for mmap)
    The zus-FS returns a set of dpp_t(s) to kernel and the Kernel does the memcopy to app buffers. The Server also specifies if those buffers should be cached on a per inode radix-tree (xarray) and if so at the next access to the same range Kernel does the copy and never dispatches to user-space
    In this mode the Server can also revoke a cached mapping when needed

5. Use of VFS page-cache
    For a very slow backing device the FS request the regular VFS page-cache. On read/write_pages() vector zuf uses option 1. above to read into page-cache instead of app-buffers directly. Only cache misses dispatch back to user-space

Have I forgotten anything?

This way the zus-FS is in control and can do the "right thing" depending on target device and FS characteristics. The interface lets us have a rich set of tools to be used.

Hope that answers your concerns
Boaz

#21 lore raw

Re: [RFC 1/7] mm: Add new vma flag VM_LOCAL_CPU

Quoted 67 lines · Boaz Harrosh
Quoted 4 lines · Miklos Szeredi

Ideally most of the complexity would be in the page cache. Not sure how ready it is to handle pmem pages?

Quoted 8 lines

The general case (non-pmem) will always have to be handled differently; you've just stated that it's much less latency sensitive
and needs async handling. Basing the design on just trying to make
it use the same mechanism (userspace copy) is flawed in my opinion, since it's suboptimal for either case.

Thanks,
Miklos

OK So I was thinking hard on all this and am changing my mind and agreeing with all that was said.

I want that the usFS plugin will have all the different options and have an easy way to tell Kernel which mode to use.

Let me summarize all the options:

1. Sync, userspace copy directly to app-buffers (current implementation)

2. Async block device operation (none pmem)
zuf owns all devices pmem and none pmem at mount time and provides a very efficient access to both. In the harddisk / ssd case as part of
an IO call
the server returns -EWOULD_BLOCK and in the background will issue a scatter_gather call through zuf.
The memory target for the IO can be pmem, directly to user-buffers
(DIO), transient
server buffers.
On completion an up call is made to ZUF to complete the IO operation
and
release the waiting application.

3. Splice and R-spilce
In the case that the IO target is not a block-device but an external
path like
network / rdma / some none block device.
Zuf already holds an internal object describing the IO context including
the
GUP app buffers. This internal object can be made the memory target of a
splice
operation.

4. Get-io_map type operation (currently implemented for mmap) The zus-FS returns a set of dpp_t(s) to kernel and the Kernel does the
memcopy
to app buffers. The Server also specifies if those buffers should be
cached
on a per inode radix-tree (xarray) and if so at the next access to the
same
range Kernel does the copy and never dispatches to user-space In this mode the Server can also revoke a cached mapping when needed

5. Use of VFS page-cache
For a very slow backing device the FS request the regular VFS
page-cache.
On read/write_pages() vector zuf uses option 1. above to read into
page-cache
instead of app-buffers directly. Only cache misses dispatch back to
user-space

Quoted 8 lines

Have I forgotten anything?

This way the zus-FS is in control and can do the "right thing" depending on target device and FS characteristics. The interface lets us have a rich set of
tools to be used.

Hope that answers your concerns

Why keep options 1 and 2? An io-map (4) type interface should cover this efficiently, shouldn't it?

I don't think page-cache is just for slow backing devices or that it needs to be a separate interface. Caches are and will always be the fastest, no matter how fast your device is. In linux the page cache seems like the most convenient place to put a pmem mapping, for example.

Of course, caches are also a big PITA when dealing with distributed filesystems. Fuse doesn't have a perfect solution for that. It's one of key areas that needs improvement.

Also I'll add one more use case that crops up often with fuse: "whole
file data mapping". Basically this means, the file's data in a
(virtual) userspace filesystem is equivalent to a file's data on an underlying (physical) filesystem. We could accelerate I/O in that
case tremendously as well as eliminating double caching. I've been
undecided what to do with it; for some time I was resisting, then saying that I'll accept patches, and at some point I'll probably do a patch myself.

Thanks,
Miklos

#22 lore raw

RFC 2/7 fs: Add the ZUF filesystem to the build + License

In reply to Boaz Harrosh

This adds the ZUF filesystem-in-user_mode module
to the fs/ build system.

Also added:
* fs/zuf/Kconfig
* fs/zuf/module.c - This file contains the LICENCE of zuf code base
* fs/zuf/Makefile - Rather empty Makefile with only module.c above

I add the fs/zuf/Makefile to demonstrate that at every patchset stage code still compiles and there are no external references outside of the code already submitted.

Off course only at the very last patch we have a working ZUF feeder

---
 fs/Kconfig       |  1 +
 fs/Makefile      |  1 +
 fs/zuf/Kconfig   | 23 +++++++++++++++++++
 fs/zuf/Makefile  | 14 ++++++++++++
 fs/zuf/module.c  | 51 +++++++++++++++++++++++++++++++++++++++++
 fs/zuf/zus_api.h | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 159 insertions(+)
 create mode 100644 fs/zuf/Kconfig
 create mode 100644 fs/zuf/Makefile
 create mode 100644 fs/zuf/module.c
 create mode 100644 fs/zuf/zus_api.h
fs/Kconfig +1 -0
index 7aee6d6..c04c454 100644
@@ -247,6 +247,7 @@ source "fs/romfs/Kconfig"
 source "fs/pstore/Kconfig"
 source "fs/sysv/Kconfig"
 source "fs/ufs/Kconfig"
+source "fs/zuf/Kconfig"
 source "fs/exofs/Kconfig"
 
 endif # MISC_FILESYSTEMS
fs/Makefile +1 -0
index ef772f1..78c13f0 100644
@@ -129,3 +129,4 @@ obj-y				+= exofs/ # Multiple modules
 obj-$(CONFIG_CEPH_FS)		+= ceph/
 obj-$(CONFIG_PSTORE)		+= pstore/
 obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
+obj-$(CONFIG_ZUF)		+= zuf/
fs/zuf/Kconfig new file +23 -0
new file mode 100644
index 0000000..a17d463
@@ -0,0 +1,23 @@
+menuconfig ZUF
+	tristate "ZUF - Zero-copy User-mode Feeder"
+	depends on BLOCK
+	depends on ZONE_DEVICE
+	select CRC16
+	select MEMCG
+	help
+	   ZUFS Kernel part.
+	   To enable say Y here.
+
+	   To compile this as a module,  choose M here: the module will be
+	   called zuf.ko
+
+if ZUF
+
+config ZUF_DEBUG
+	bool "ZUF: enable debug subsystems use"
+	depends on ZUF
+	default n
+	help
+	  INTERNAL QA USE ONLY!!! DO NOT USE!!!
+
+endif
fs/zuf/Makefile new file +14 -0
new file mode 100644
index 0000000..7e4e51f
@@ -0,0 +1,14 @@
+#
+# ZUF: Zero-copy User-mode Feeder
+#
+# Copyright (c) 2018 NetApp Inc. All rights reserved.
+#
+# ZUFS-License: GPL-2.0 OR BSD-3-Clause. See module.c for LICENSE details.
+#
+# Makefile for the Linux zufs Kernel Feeder.
+#
+
+obj-$(CONFIG_ZUF) += zuf.o
+
+# Main FS
+zuf-y += module.o
fs/zuf/module.c new file +51 -0

51 lines not on this page · raw message

fs/zuf/zus_api.h new file +69 -0

69 lines not on this page · raw message

Signature · 1 line
2.5.5
#23 lore raw

Re: [RFC 2/7] fs: Add the ZUF filesystem to the build + License

Quoted 29 lines · Boaz Harrosh

This adds the ZUF filesystem-in-user_mode module to the fs/ build system.

Also added:
* fs/zuf/Kconfig
* fs/zuf/module.c - This file contains the LICENCE of zuf code base
* fs/zuf/Makefile - Rather empty Makefile with only module.c above

I add the fs/zuf/Makefile to demonstrate that at every patchset stage code still compiles and there are no external references outside of the code already submitted.

Off course only at the very last patch we have a working ZUF feeder

fs/zuf/zus_api.h new file +11 -0
+/*
+ * Version rules:
+ *   This is the zus-to-zuf API version. And not the Filesystem
+ * on disk structures versions. These are left to the FS-plugging
+ * to supply and check.
+ * Specifically any of the API structures and constants found in this
+ * file.
+ * If the changes are made in a way backward compatible with old
+ * user-space, MINOR is incremented. Else MAJOR is incremented.
+ *
+ * We believe that the zus Server application comes with the

fs/zuf/zus_api.h line 43

Quoted 10 lines
fs/zuf/zus_api.h new file +10 -0
+ * Distro and should be dependent on the Kernel package.
+ * The more stable ABI is between the zus Server and its FS plugins.
+ * Because of the intimate relationships in the zuf-core behavior
+ * We would also like zus Server to be signed by the running Kernel's
+ * make crypto key and checked before load because of the Security
+ * nature of an FS provider.
+ */
+#define ZUFS_MINORS_PER_MAJOR	1024
+#define ZUFS_MAJOR_VERSION 1
+#define ZUFS_MINOR_VERSION 0

I haven't really been following this development, but my recommendation would be to use feature flags (e.g. at least __u64 compat, __u64 incompat) for the API and separately for the disk format, rather than using version numbers. This makes it clear what "version" relates to a specific feature, and also allows *removal* of features if they turn out to be a bad idea. With version numbers you can only ever *add* features, and have to keep support for every old feature added.

Also, having separate feature flags allows independent development of new features, and doesn't require that feature X has to be in version N or it will break for anyone using/testing that feature outside of the main tree.

This has worked for 25 years for ext2/3/4 and 15 years for Lustre. ZFS has a slightly more complex feature flags, distinguishing between features that _could_ be used (i.e. enabled at format time or by the administrator), and features that _are_ used (with a refcount). That avoids gratuitous incompatibility if some feature is enabled, but not actually used (e.g. ext4 files over 2TB), and also allows removing that incompatibility if the feature is no longer used (e.g. all > 2TB files are deleted).

Cheers, Andreas

#24 lore raw

Re: [RFC 2/7] fs: Add the ZUF filesystem to the build + License

On 13/03/18 22:16, Andreas Dilger wrote:
<>

Quoted 17 lines

fs/zuf/zus_api.h line 43

Quoted 4 lines

+ */

fs/zuf/zus_api.h new file +3 -0
+#define ZUFS_MINORS_PER_MAJOR	1024
+#define ZUFS_MAJOR_VERSION 1
+#define ZUFS_MINOR_VERSION 0

I haven't really been following this development, but my recommendation would be to use feature flags (e.g. at least __u64 compat, __u64 incompat) for the API and separately for the disk format, rather than using version numbers. This makes it clear what "version" relates to a specific feature, and also allows *removal* of features if they turn out to be a bad idea. With version numbers you can only ever *add* features, and have to keep support for every old feature added.

Also, having separate feature flags allows independent development of new features, and doesn't require that feature X has to be in version N or it will break for anyone using/testing that feature outside of the main tree.

Quoted 8 lines

This has worked for 25 years for ext2/3/4 and 15 years for Lustre. ZFS has a slightly more complex feature flags, distinguishing between features that _could_ be used (i.e. enabled at format time or by the administrator), and features that _are_ used (with a refcount). That avoids gratuitous incompatibility if some feature is enabled, but not actually used (e.g. ext4 files over 2TB), and also allows removing that incompatibility if the feature is no longer used (e.g. all > 2TB files are deleted).

Yes thank you. As you can see at this RFC stage I have not even put any code to enforce the ABI/API versioning yet. Exactly because I don't like it as you explained. I will think about your suggestion and see. This is not on disk stuff. This is more the communication channel between ZUF<=>ZUS. Though there are a couple on disk stuff.
(The on disk things are all hidden from here inside the usermode FS plugin)

The thing is that I want to work a system with the distro's that the ZUF<=>ZUS ABI can freely change, by forcing the zusd package be dependent on the kernel package. And it be signed by the Kernel's make key. Meaning it will only run against the kernel it was compiled against.

And keep the stable ABI with feature and versioning between the ZUSD<=>zusFS-plugin(s)
We'll have to see

Thanks
Boaz

Quoted 3 lines

Cheers, Andreas

#25 lore raw

Re: [RFC 2/7] fs: Add the ZUF filesystem to the build + License

Quoted 36 lines · Boaz Harrosh

On 13/03/18 22:16, Andreas Dilger wrote:
<>

Quoted 17 lines

fs/zuf/zus_api.h line 43

Quoted 4 lines

+ */

fs/zuf/zus_api.h new file +3 -0
+#define ZUFS_MINORS_PER_MAJOR	1024
+#define ZUFS_MAJOR_VERSION 1
+#define ZUFS_MINOR_VERSION 0

I haven't really been following this development, but my recommendation would be to use feature flags (e.g. at least __u64 compat, __u64 incompat) for the API and separately for the disk format, rather than using version numbers. This makes it clear what "version" relates to a specific feature, and also allows *removal* of features if they turn out to be a bad idea. With version numbers you can only ever *add* features, and have to keep support for every old feature added.

Also, having separate feature flags allows independent development of new features, and doesn't require that feature X has to be in version N or it will break for anyone using/testing that feature outside of the main tree.

Quoted 8 lines

This has worked for 25 years for ext2/3/4 and 15 years for Lustre. ZFS has a slightly more complex feature flags, distinguishing between features that _could_ be used (i.e. enabled at format time or by the administrator), and features that _are_ used (with a refcount). That avoids gratuitous incompatibility if some feature is enabled, but not actually used (e.g. ext4 files over 2TB), and also allows removing that incompatibility if the feature is no longer used (e.g. all > 2TB files are deleted).

Yes thank you. As you can see at this RFC stage I have not even put any code to enforce the ABI/API versioning yet. Exactly because I don't like it as you explained. I will think about your suggestion and see. This is not on disk stuff. This is more the communication channel between ZUF<=>ZUS. Though there are a couple on disk stuff.
(The on disk things are all hidden from here inside the usermode FS plugin)

Quoted 4 lines

The thing is that I want to work a system with the distro's that the ZUF<=>ZUS ABI can freely change, by forcing the zusd package be dependent on the kernel package. And it be signed by the Kernel's make key. Meaning it will only run against the kernel it was compiled against.

That is a major pain, and even the distros are doing things like weak module symbol versions so that external kernel modules do not need to be rebuilt for every minor kernel update.

Quoted 6 lines

And keep the stable ABI with feature and versioning between the ZUSD<=>zusFS-plugin(s)
We'll have to see

Thanks
Boaz

Cheers, Andreas

#26 lore raw

Re: [RFC 2/7] fs: Add the ZUF filesystem to the build + License

Quoted 37 lines · Andreas Dilger
Quoted 36 lines · Boaz Harrosh

On 13/03/18 22:16, Andreas Dilger wrote:
<>

Quoted 17 lines

fs/zuf/zus_api.h line 43

Quoted 4 lines

+ */

fs/zuf/zus_api.h new file +3 -0
+#define ZUFS_MINORS_PER_MAJOR	1024
+#define ZUFS_MAJOR_VERSION 1
+#define ZUFS_MINOR_VERSION 0

I haven't really been following this development, but my recommendation would be to use feature flags (e.g. at least __u64 compat, __u64 incompat) for the API and separately for the disk format, rather than using version numbers. This makes it clear what "version" relates to a specific feature, and also allows *removal* of features if they turn out to be a bad idea. With version numbers you can only ever *add* features, and have to keep support for every old feature added.

Also, having separate feature flags allows independent development of new features, and doesn't require that feature X has to be in version N or it will break for anyone using/testing that feature outside of the main tree.

Quoted 8 lines

This has worked for 25 years for ext2/3/4 and 15 years for Lustre. ZFS has a slightly more complex feature flags, distinguishing between features that _could_ be used (i.e. enabled at format time or by the administrator), and features that _are_ used (with a refcount). That avoids gratuitous incompatibility if some feature is enabled, but not actually used (e.g. ext4 files over 2TB), and also allows removing that incompatibility if the feature is no longer used (e.g. all > 2TB files are deleted).

Yes thank you. As you can see at this RFC stage I have not even put any code to enforce the ABI/API versioning yet. Exactly because I don't like it as you explained. I will think about your suggestion and see. This is not on disk stuff. This is more the communication channel between ZUF<=>ZUS. Though there are a couple on disk stuff.
(The on disk things are all hidden from here inside the usermode FS plugin)

Quoted 9 lines
Quoted 4 lines

The thing is that I want to work a system with the distro's that the ZUF<=>ZUS ABI can freely change, by forcing the zusd package be dependent on the kernel package. And it be signed by the Kernel's make key. Meaning it will only run against the kernel it was compiled against.

That is a major pain, and even the distros are doing things like weak module symbol versions so that external kernel modules do not need to be rebuilt for every minor kernel update.

OK I get it. So yes let me think about it. we have two structures and then all these IOCTL definitions.

For the two structures: struct multi_device and struct zus_inode we can have the above version as an indicator of compatibility,

And with the IOCTLs each time any structure change we can define a new IOCTL constant and/or operation number. Or put the sizeof at the header.

I'll see what's simple enough to do
Boaz

Quoted 13 lines
Quoted 6 lines

And keep the stable ABI with feature and versioning between the ZUSD<=>zusFS-plugin(s)
We'll have to see

Thanks
Boaz

Cheers, Andreas

#27 lore raw

RFC 3/7 zuf: Preliminary Documentation

In reply to Boaz Harrosh

Adding Documentation/filesystems/zufs.txt

---
 Documentation/filesystems/zufs.txt | 351 +++++++++++++++++++++++++++++++++++++
 1 file changed, 351 insertions(+)
 create mode 100644 Documentation/filesystems/zufs.txt
Documentation/filesystems/zufs.txt new file +351 -0
new file mode 100644
index 0000000..779f14b
@@ -0,0 +1,351 @@
+ZUFS - Zero-copy User-mode FileSystem
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Trees:
+	git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+	git clone https://github.com/NetApp/zufs-zus -b zus-github
+
+patches, comments, questions, requests to:
+	[email protected]
+
+Introduction:
+~~~~~~~~~~~~~
+
+ZUFS - stands for Zero-copy User-mode FS
+▪ It is geared towards true zero copy end to end of both data and meta data.
+▪ It is geared towards very *low latency*, very high CPU locality, lock-less
+  parallelism.
+▪ Synchronous operations
+▪ Numa awareness
+
+ ZUFS is a, from scratch, implementation of a filesystem-in-user-space, which
+tries to address the above goals. It is aimed for pmem based FSs. But can easily
+support any other type of FSs that can utilize x10 latency and parallelism
+improvements.
+
+Glossary and names:
+~~~~~~~~~~~~~~~~~~~
+
+ZUF - Zero-copy User-mode Feeder
+  zuf.ko is the Kernel VFS component. Its job is to interface with the Kernel
+  VFS and dispatch commands to a User-mode application Server.
+  Uptodate code is found at:
+	git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+
+ZUS - Zero-copy User-mode Server
+  zufs utilizes a User-mode server application. That takes care of the detailed
+  communication protocol and correctness with the Kernel.
+  In turn it utilizes many zusFS Filesystem plugins to implement the actual
+  on disc Filesystem.
+  Uptodate code is found at:
+	git clone https://github.com/NetApp/zufs-zus -b zus-github
+
+zusFS - FS plugins
+  These are .so loadable modules that implement one or more Filesystem-types
+  (-t xyz).
+  The zus server communicates with the plugin via a set of function vectors
+  for the different operations. And establishes communication via defined
+  structures.
+
+Filesystem-type:
+  At startup zus registers with the Kernel one or more Filesystem-type(s)
+  Associated with the type is a 4 letter type-name (-t fstn) different
+  info about the fs, like a magic number and so on.
+  One Server can support many FS-types, in turn each FS-type can mount
+  multiple super-blocks, each supporting multiple devices.
+
+Device-Table (DT) - A zufs FS can support multiple devices
+  ZUF in Kernel may receive, like any mount command a block-device or none.
+  For the former if the specified FS-types states so in a special field.
+  The mount will look for a Device table. A list of devices in a specific
+  order sitting at some offset on the block-device. The system will then
+  proceed to open and own all these devices and associate them to the mounting
+  super-block.
+  If FS-type specifies a -1 at DT_offset then there is no device table
+  and a DT of a single device is created. (If we have no devices, none
+  is specified than we operate without any block devices. (Mount options give
+  some indication of the storage information)
+  The device table has special consideration for pmem devices and will
+  present the all linear array of devices to zus, as one flat mmap space.
+  Alternatively all none pmem devices are also provided an interface
+  with facility of data movement from pmem to a slower device.
+  A detailed NUMA info is exported to the Server for maximum utilization.
+
+pmem:
+  Multiple pmem devices are presented to the server as a single
+  linear file mmap. Something like /dev/dax. But it is strictly
+  available only to that specific super-block that owns it.
+
+dpp_t - Dual port pointer type
+  At some points in the protocol there are objects that return from zus
+  (The Server) to the Kernel via a dpp_t. This is a special kind of pointer
+  It is actually an offset 8 bytes aligned with the 3 low bits specifying
+  a pool code: [offset = dpp_t & ~0x7] [pool = dpp_t & 0x7]
+  pool == 0 means the offset is in pmem who's management is by zuf and
+  a full easy access is provided for zus.
+
+  pool != 0 Is a pre-established tempfs file (up to 6 such files) where
+  the zus has an mmap on the file and the Kernel can access that data
+  via an offset into the file.
+  All dpp_t objects life time rules are strictly defined.
+  Mainly the primary use of dpp_t is the on-pmem inode structure. Both
+  zus and zuf can access and change this structure. On any modification
+  the zus is called so to be notified of any changes, persistence.
+  More such objects are: Symlinks, xattrs, mmap-data-blocks etc...
+
+Relay-wait-object:
+  communication between Kernel and server are done via zus-threads that
+  sleep in Kernel (inside an IOCTL) and wait for commands. Once received
+  the IOCTL returns operation id executed and the return info is returned via
+  a new IOCTL call, which then waits for the next operation.
+  To wake up the sleeping thread we use a Relay-wait-object. Currently
+  it is two waitqueue_head(s) back to back.
+  In future we should investigate the use of that special binder object
+  that releases its thread time slice to the other thread without going through
+  the scheduler.
+
+ZT-threads-array:
+  The novelty of the zufs is the ZT-threads system. One thread or more is
+  pre-created for each active core in the system.
+  ▪ The thread is AFFINITY set for that single core only.
+  ▪ Special communication file per ZT (O_TMPFILE + IOCTL_ZUFS_INIT)
+    At initialization the ZT thread communicates through a ZT_INIT ioctl
+    and registers as the handler of that core (Channel)
+  ▪ ZT-vma - Mmap 4M vma zero copy communication area per ZT
+    Pre allocated vma is created into which will be mapped the application
+    or Kernel buffers for the current operation.
+  ▪ IOCTL_ZU_WAIT_OPT – threads sleeps in Kernel waiting for an operation
+    via the IOCTL_ZU_WAIT_OPT call. supplying a 4k communication buffer
+
+  ▪ On an operation dispatch current CPU's ZT is selected, app pages mapped
+    into the ZT-vma. Server thread released with an operation to execute.
+  ▪ After execution, ZT returns to kernel (IOCTL_ZU_WAIT_OPT), app is released,
+    Server wait for new operation on that CPU.
+
+ZUS-mount-thread:
+  The system utilizes a single mount thread. (This thread is not affinity to any
+  core).
+  ▪ It will first Register all FS-types supported by this Server (By calling
+    all zusFS plugins to register their supported types). Once done
+  ▪ As above the thread sleeps in Kernel via the IOCTL_ZU_MOUNT call.
+  ▪ When the Kernel receives a mount request (vfs calles the fs_type->mount opt)
+    a mount is dispatched back to zus.
+  ▪ NOTE: That only on very first mount the above ZT-threads-array is created
+    the same array is then used for all super-blocks in the system
+  ▪ As part of the mount command in the context of this same mount-thread
+    a call to IOCTL_ZU_GRAB_PMEM will establish an interface to the pmem
+    Associated with this super_block
+  ▪ On return (like above a new call to IOCTL_ZU_MOUNT will return info of the
+    mount before sleeping in kernel waiting for a new dispatch. All SB info
+    is provided to zuf, including the root inode info. Kernel then proceeds
+    to complete the mount call.
+  ▪ NOTE that since there is a single mount thread all FS-registration
+    super_block and pmem management are lockless.
+  
+Philosophy of operations:
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. [zuf-root]
+
+On module load  (zuf.ko) A special pseudo FS is mounted on /sys/fs/zuf. This is
+called zuf-root.
+The zuf-root has no visible files. All communication is done via special-files.
+special-files are open(O_TMPFILE) and establish a special role via an
+IOCTL.
+All communications with the server are done via the zuf-root. Each root owns
+many FS-types and each FS-type owns many super-blocks of this type. All Sharing
+the same communication channels.
+Since all FS-type Servers live in the same zus application address space, at
+times. If the administrator wants to separate between different servers, he/she
+can mount a new zuf-root and point a new server instance on that new mount,
+registering other FS-types on that other instance. The all communication array
+will then be duplicated as well.
+(Otherwise pointing a new server instance on a busy root will return an error)
+
+2. [zus server start]
+  ▪ On load all configured zusFS plugins are loaded.
+  ▪ The Server starts by starting a single mount thread.
+  ▪ It than proceeds to register with Kernel all FS-types it will support.
+    (This is done on the single mount thread, so all FS-registration and
+     mount/umount operate in a single thread and therefor need not any locks)
+  ▪ Sleeping in the Kernel on a special-file of that zuf-root. waiting for a mount
+    command.
+
+3. [mount -t xyz]
+  [In Kernel]
+  ▪ If xyz was registered above as part of the Server startup. the regular
+    mount command will come to the zuf module with a zuf_mount() call. with
+    the xyz-FS-info. In turn this points to a zuf-root.
+  ▪ Code than proceed to load a device-table of devices as  specified above.
+    It then establishes an md object with a specific pmem_id.
+  ▪ It proceeds to call mount_bdev. Always with the same main-device
+    thous fully sporting automatic bind mounts. Even if different
+    devices are given to the mount command.
+  ▪ In zuf_fill_super it will then dispatch (awaken) the mount thread
+    specifying two parameters. One the FS-type to mount, and then
+    the pmem_id Associated with this super_block.
+
+  [In zus]
+  ▪ A zus_super_block_info is allocated.
+  ▪ zus calls PMEM_GRAB(pmem_id) to establish a direct mapping to its
+    pmem devices. On return we have full access to our PMEM
+
+  ▪ ZT-threads-array
+    If this is the first mount the all ZT-threads-array is created and
+    established. The mount thread will wait until all zt-threads finished
+    initialization and ready to rock.
+  ▪ Root-zus_inode is loaded and is returned to kernel
+  ▪ More info about the mount like block sizes and so on are returned to kernel.
+
+  [In Kernel]
+   The zuf_fill_super is finalized vectors established and we have a new
+   super_block ready for operations.
+
+4. An FS operation like create or WRITE/READ and so on arrives from application
+   via VFS. Eventually an Operation is dispatched to zus:
+   ▪ A special per-operation descriptor is filled up with all parameters.
+   ▪ A current CPU channel is grabbed. the operation descriptor is put on
+     that channel (ZT). Including get_user_pages or Kernel-pages associated
+     with this OPT.
+   ▪ The ZT is awaken, app thread put to sleep.
+   ▪ In ZT context pages are mapped to that ZT-vma. This is so we are sure
+     the map is only on a single core. And no other core's TLB is affected.
+     (This here is the all performance secret)
+   ▪ ZT thread is returned to user-space.
+   ▪ In ZT context the zus Server calls the appropriate zusFS->operation
+     vector. Output params filled.
+   ▪ zus calls again with an IOCTL_ZU_WAIT_OPT with the same descriptor
+     to return the requested info.
+   ▪ At Kernel (zuf) the app thread is awaken with the results, and the
+     ZT thread goes back to sleep waiting a new operation.
+     
+   ZT rules:
+       A ZT thread must not return back to Kernel. One exception is locks
+   if needed it might sleep waiting for a lock. In which case we will see that
+   the same CPU channel is reentered via another application and/or thread.
+   But now that CPU channel is taken.  What we do is we utilize a few channels
+   (ZTs) per core and the threads may grab another channel. But this only
+   postpones the problem on a busy contended system, all such channels will be
+   consumed. If all channels are taken the application thread is put on a busy
+   scheduling wait until a channel can be grabbed.
+   Therefor Server must not sleep on a ZT. If it needs such a sleeping operation
+   it will return -EAGAIN to zuf. The app is kept sleeping the operation is put
+   on an asynchronous Q and the ZT freed for foreground operation. At some point
+   when the server completes the delayed operation it will complete notify
+   the Kernel with a special async cookie. And the app will be awakened.
+   (Here too we utilize pre allocated asyc channels and vmas. If all channels
+    are busy, application is kept sleeping waiting its free slot turn)
+
+4. On umount the operation is reversed and all resources are torn down.
+5. In case of an application or Server crash, all resources are Associated
+   with files, on file_release these resources are caught and freed.
+
+Objects and life-time
+~~~~~~~~~~~~~~~~~~~~~
+
+Each Kernel object type has an assosiated zus Server object type who's life
+time is governed by the life-time of the Kernel object. Therefor the Server's
+job is easy because it need not establish any object caches / hashes and so on.
+
+Inside zus all objects are allocated by the zusFS plugin. So in turn it can
+allocate a bigger space for its own private data and access it via the
+container_off() coding pattern. So when I say below a zus-object I mean both
+zus public part + zusFS private part of the same object.
+
+All operations return a UM pointer that are OPEC the the Kernel code, they
+are just a cookie which is returned back to zus, when needed.
+At times when we want the Kernel to have direct access to a zus object like
+zus_inode, along with the cookie we also return a dpp_t, with a defined structure.
+
+Kernel object 			| zus object 		| Kernel access (via dpp_t)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+zuf_fs_type
+	file_system_type	| zus_fs_info		| no
+
+zuf_sb_info
+	super_block		| zus_sb_info		| no
+	
+zuf_inode_info			|			|
+	vfs_inode		| zus_inode_info	| no
+	zus_inode *		| 	zus_inode *	| yes
+	synlink *		|	char-array	| yes
+	xattr**			|	zus_xattr	| yes
+
+When a Kernel object's time is to die, a final call to zus is
+dispatched so the associated object can also be freed. Which means
+that on memory pressure when object caches are evicted also the zus
+memory resources are freed.
+
+
+How to use zufs:
+~~~~~~~~~~~~~~~~
+
+The most updated documentation of how to use the latest code bases
+is the script (set of scripts) at fs/do-zu/zudo on the zus git tree
+
+We the developers at Netapp use this script to mount and test our
+latest code. So any new Secret will be found in these scripts. Please
+read them as the ultimate source of how to operate things.
+
+TODO: We are looking for exports in system-d and udev to properly
+integrate these tools into a destro.
+
+We assume you cloned these git trees:
+[]$ mkdir zufs; cd zufs
+[]$ git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+[]$ git clone https://github.com/NetApp/zufs-zuf -b zus-github
+
+This will create the following trees
+zufs/zus - Source code for Server
+zufs/zuf - Linux Kernel source tree to compile and install on your machine
+
+Also specifically:
+zufs/zus/fs/do-zu/zudo - script Documenting how to run things
+
+[]$ cd zuf
+
+First time
+[] ../zus/fs/do-zu/zudo
+this will create a file:
+	../zus/fs/do-zu/zu.conf
+
+Edit this file for your environment. Devices, mount-point and so on.
+On first run an example file will be created for you. Fill in the
+blanks. Most params can stay as is in most cases
+
+Now lest start running:
+
+[1]$ ../zus/fs/do-zu/zudo mkfs
+This will run the proper mkfs command selected at zu.conf file
+with the proper devices.
+
+[2]$ ../zus/fs/do-zu/zudo zuf-insmod
+This loads the zuf.ko module
+
+[3]$ ../zus/fs/do-zu/zudo zuf-root
+This mounts the zuf-root FS above on /sys/fs/zuf (automatically created above)
+
+[4]$ ../zus/fs/do-zu/zudo zus-up
+This runs the zus daemon in the background
+
+[5]$ ../zus/fs/do-zu/zudo mount
+This mount the mkfs FS above on the specified dir in zu.conf
+
+To run all the 5 commands above at once do:
+[]$ ../zus/fs/do-zu/zudo up
+
+To undo all the above in reverse order do:
+[]$ ../zus/fs/do-zu/zudo down
+
+And the most magic command is:
+[]$ ../zus/fs/do-zu/zudo again
+Will do a "down", then update-mods, then "up"
+(update-mods is a special script to copy the latest compiled binaries)
+
+Now you are ready for some:
+[]$ ../zus/fs/do-zu/zudo xfstest
+xfstests is assumed to be installed in the regular /opt/xfstests dir
+
+Again please see inside the scripts what each command does
+these scripts are the ultimate Documentation, do not believe
+anything I'm saying here. (Because it is outdated by now)
Signature · 1 line
2.5.5
reviewer
#28 lore raw

Re: [RFC 3/7] zuf: Preliminary Documentation

Hi,

Just a few questions. Very little editing. :)

Quoted 57 lines · Boaz Harrosh

Adding Documentation/filesystems/zufs.txt

---
 Documentation/filesystems/zufs.txt | 351 +++++++++++++++++++++++++++++++++++++
 1 file changed, 351 insertions(+)
 create mode 100644 Documentation/filesystems/zufs.txt
Documentation/filesystems/zufs.txt new file +42 -0
new file mode 100644
index 0000000..779f14b
@@ -0,0 +1,351 @@
+ZUFS - Zero-copy User-mode FileSystem
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Trees:
+	git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+	git clone https://github.com/NetApp/zufs-zus -b zus-github
+
+patches, comments, questions, requests to:
+	[email protected]
+
+Introduction:
+~~~~~~~~~~~~~
+
+ZUFS - stands for Zero-copy User-mode FS
+▪ It is geared towards true zero copy end to end of both data and meta data.
+▪ It is geared towards very *low latency*, very high CPU locality, lock-less
+  parallelism.
+▪ Synchronous operations
+▪ Numa awareness
+
+ ZUFS is a, from scratch, implementation of a filesystem-in-user-space, which
+tries to address the above goals. It is aimed for pmem based FSs. But can easily
+support any other type of FSs that can utilize x10 latency and parallelism
+improvements.
+
+Glossary and names:
+~~~~~~~~~~~~~~~~~~~
+
+ZUF - Zero-copy User-mode Feeder
+  zuf.ko is the Kernel VFS component. Its job is to interface with the Kernel
+  VFS and dispatch commands to a User-mode application Server.
+  Uptodate code is found at:
+	git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+
+ZUS - Zero-copy User-mode Server
+  zufs utilizes a User-mode server application. That takes care of the detailed
+  communication protocol and correctness with the Kernel.
+  In turn it utilizes many zusFS Filesystem plugins to implement the actual
+  on disc Filesystem.
+  Uptodate code is found at:
+	git clone https://github.com/NetApp/zufs-zus -b zus-github
+

Documentation/filesystems/zufs.txt line 52

Quoted 10 lines
Documentation/filesystems/zufs.txt new file +10 -0
+zusFS - FS plugins
+  These are .so loadable modules that implement one or more Filesystem-types
+  (-t xyz).
+  The zus server communicates with the plugin via a set of function vectors
+  for the different operations. And establishes communication via defined
+  structures.
+
+Filesystem-type:
+  At startup zus registers with the Kernel one or more Filesystem-type(s)
+  Associated with the type is a 4 letter type-name (-t fstn) different

(?) is a unique 4-letter type-name

Quoted 5 lines
Documentation/filesystems/zufs.txt new file +5 -0
+  info about the fs, like a magic number and so on.
+  One Server can support many FS-types, in turn each FS-type can mount
+  multiple super-blocks, each supporting multiple devices.
+
+Device-Table (DT) - A zufs FS can support multiple devices

Documentation/filesystems/zufs.txt line 67

Quoted 10 lines
Documentation/filesystems/zufs.txt new file +10 -0
+  ZUF in Kernel may receive, like any mount command a block-device or none.
+  For the former if the specified FS-types states so in a special field.
+  The mount will look for a Device table. A list of devices in a specific
+  order sitting at some offset on the block-device. The system will then
+  proceed to open and own all these devices and associate them to the mounting
+  super-block.
+  If FS-type specifies a -1 at DT_offset then there is no device table
+  and a DT of a single device is created. (If we have no devices, none
+  is specified than we operate without any block devices. (Mount options give
+  some indication of the storage information)

missing one ')'

Documentation/filesystems/zufs.txt line 70

Quoted 3 lines
Documentation/filesystems/zufs.txt new file +3 -0
+  The device table has special consideration for pmem devices and will
+  present the all linear array of devices to zus, as one flat mmap space.
+  Alternatively all none pmem devices are also provided an interface

all known (?)

Quoted 9 lines
Documentation/filesystems/zufs.txt new file +9 -0
+  with facility of data movement from pmem to a slower device.
+  A detailed NUMA info is exported to the Server for maximum utilization.
+
+pmem:
+  Multiple pmem devices are presented to the server as a single
+  linear file mmap. Something like /dev/dax. But it is strictly
+  available only to that specific super-block that owns it.
+
+dpp_t - Dual port pointer type

Documentation/filesystems/zufs.txt line 89

Quoted 10 lines
Documentation/filesystems/zufs.txt new file +10 -0
+  At some points in the protocol there are objects that return from zus
+  (The Server) to the Kernel via a dpp_t. This is a special kind of pointer
+  It is actually an offset 8 bytes aligned with the 3 low bits specifying
+  a pool code: [offset = dpp_t & ~0x7] [pool = dpp_t & 0x7]
+  pool == 0 means the offset is in pmem who's management is by zuf and
+  a full easy access is provided for zus.
+
+  pool != 0 Is a pre-established tempfs file (up to 6 such files) where
+  the zus has an mmap on the file and the Kernel can access that data
+  via an offset into the file.

so non-zero pool [dpp_t & 0x7] can be a value of 1 - 7, and above says up to 6 such tempfs files. What is the other pool value used for?

Documentation/filesystems/zufs.txt line 100

Quoted 11 lines
Documentation/filesystems/zufs.txt new file +11 -0
+  All dpp_t objects life time rules are strictly defined.
+  Mainly the primary use of dpp_t is the on-pmem inode structure. Both
+  zus and zuf can access and change this structure. On any modification
+  the zus is called so to be notified of any changes, persistence.
+  More such objects are: Symlinks, xattrs, mmap-data-blocks etc...
+
+Relay-wait-object:
+  communication between Kernel and server are done via zus-threads that
+  sleep in Kernel (inside an IOCTL) and wait for commands. Once received
+  the IOCTL returns operation id executed and the return info is returned via
+  a new IOCTL call, which then waits for the next operation.

Does that say 2 IOCTLs per command? One to start it and one to fetch return info?

Quoted 28 lines
Documentation/filesystems/zufs.txt new file +28 -0
+  To wake up the sleeping thread we use a Relay-wait-object. Currently
+  it is two waitqueue_head(s) back to back.
+  In future we should investigate the use of that special binder object
+  that releases its thread time slice to the other thread without going through
+  the scheduler.
+
+ZT-threads-array:
+  The novelty of the zufs is the ZT-threads system. One thread or more is
+  pre-created for each active core in the system.
+  ▪ The thread is AFFINITY set for that single core only.
+  ▪ Special communication file per ZT (O_TMPFILE + IOCTL_ZUFS_INIT)
+    At initialization the ZT thread communicates through a ZT_INIT ioctl
+    and registers as the handler of that core (Channel)
+  ▪ ZT-vma - Mmap 4M vma zero copy communication area per ZT
+    Pre allocated vma is created into which will be mapped the application
+    or Kernel buffers for the current operation.
+  ▪ IOCTL_ZU_WAIT_OPT – threads sleeps in Kernel waiting for an operation
+    via the IOCTL_ZU_WAIT_OPT call. supplying a 4k communication buffer
+
+  ▪ On an operation dispatch current CPU's ZT is selected, app pages mapped
+    into the ZT-vma. Server thread released with an operation to execute.
+  ▪ After execution, ZT returns to kernel (IOCTL_ZU_WAIT_OPT), app is released,
+    Server wait for new operation on that CPU.
+
+ZUS-mount-thread:
+  The system utilizes a single mount thread. (This thread is not affinity to any
+  core).
+  ▪ It will first Register all FS-types supported by this Server (By calling

Documentation/filesystems/zufs.txt line 138

Quoted 10 lines
Documentation/filesystems/zufs.txt new file +10 -0
+    all zusFS plugins to register their supported types). Once done
+  ▪ As above the thread sleeps in Kernel via the IOCTL_ZU_MOUNT call.
+  ▪ When the Kernel receives a mount request (vfs calles the fs_type->mount opt)
+    a mount is dispatched back to zus.
+  ▪ NOTE: That only on very first mount the above ZT-threads-array is created
+    the same array is then used for all super-blocks in the system
+  ▪ As part of the mount command in the context of this same mount-thread
+    a call to IOCTL_ZU_GRAB_PMEM will establish an interface to the pmem
+    Associated with this super_block
+  ▪ On return (like above a new call to IOCTL_ZU_MOUNT will return info of the

missing ')' somewhere.

Quoted 32 lines
Documentation/filesystems/zufs.txt new file +32 -0
+    mount before sleeping in kernel waiting for a new dispatch. All SB info
+    is provided to zuf, including the root inode info. Kernel then proceeds
+    to complete the mount call.
+  ▪ NOTE that since there is a single mount thread all FS-registration
+    super_block and pmem management are lockless.
+  
+Philosophy of operations:
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. [zuf-root]
+
+On module load  (zuf.ko) A special pseudo FS is mounted on /sys/fs/zuf. This is
+called zuf-root.
+The zuf-root has no visible files. All communication is done via special-files.
+special-files are open(O_TMPFILE) and establish a special role via an
+IOCTL.
+All communications with the server are done via the zuf-root. Each root owns
+many FS-types and each FS-type owns many super-blocks of this type. All Sharing
+the same communication channels.
+Since all FS-type Servers live in the same zus application address space, at
+times. If the administrator wants to separate between different servers, he/she
+can mount a new zuf-root and point a new server instance on that new mount,
+registering other FS-types on that other instance. The all communication array
+will then be duplicated as well.
+(Otherwise pointing a new server instance on a busy root will return an error)
+
+2. [zus server start]
+  ▪ On load all configured zusFS plugins are loaded.
+  ▪ The Server starts by starting a single mount thread.
+  ▪ It than proceeds to register with Kernel all FS-types it will support.
+    (This is done on the single mount thread, so all FS-registration and
+     mount/umount operate in a single thread and therefor need not any locks)

Documentation/filesystems/zufs.txt line 180

Quoted 10 lines
Documentation/filesystems/zufs.txt new file +10 -0
+  ▪ Sleeping in the Kernel on a special-file of that zuf-root. waiting for a mount
+    command.
+
+3. [mount -t xyz]
+  [In Kernel]
+  ▪ If xyz was registered above as part of the Server startup. the regular
+    mount command will come to the zuf module with a zuf_mount() call. with
+    the xyz-FS-info. In turn this points to a zuf-root.
+  ▪ Code than proceed to load a device-table of devices as  specified above.
+    It then establishes an md object with a specific pmem_id.
                              md ??
Quoted 65 lines
Documentation/filesystems/zufs.txt new file +65 -0
+  ▪ It proceeds to call mount_bdev. Always with the same main-device
+    thous fully sporting automatic bind mounts. Even if different
+    devices are given to the mount command.
+  ▪ In zuf_fill_super it will then dispatch (awaken) the mount thread
+    specifying two parameters. One the FS-type to mount, and then
+    the pmem_id Associated with this super_block.
+
+  [In zus]
+  ▪ A zus_super_block_info is allocated.
+  ▪ zus calls PMEM_GRAB(pmem_id) to establish a direct mapping to its
+    pmem devices. On return we have full access to our PMEM
+
+  ▪ ZT-threads-array
+    If this is the first mount the all ZT-threads-array is created and
+    established. The mount thread will wait until all zt-threads finished
+    initialization and ready to rock.
+  ▪ Root-zus_inode is loaded and is returned to kernel
+  ▪ More info about the mount like block sizes and so on are returned to kernel.
+
+  [In Kernel]
+   The zuf_fill_super is finalized vectors established and we have a new
+   super_block ready for operations.
+
+4. An FS operation like create or WRITE/READ and so on arrives from application
+   via VFS. Eventually an Operation is dispatched to zus:
+   ▪ A special per-operation descriptor is filled up with all parameters.
+   ▪ A current CPU channel is grabbed. the operation descriptor is put on
+     that channel (ZT). Including get_user_pages or Kernel-pages associated
+     with this OPT.
+   ▪ The ZT is awaken, app thread put to sleep.
+   ▪ In ZT context pages are mapped to that ZT-vma. This is so we are sure
+     the map is only on a single core. And no other core's TLB is affected.
+     (This here is the all performance secret)
+   ▪ ZT thread is returned to user-space.
+   ▪ In ZT context the zus Server calls the appropriate zusFS->operation
+     vector. Output params filled.
+   ▪ zus calls again with an IOCTL_ZU_WAIT_OPT with the same descriptor
+     to return the requested info.
+   ▪ At Kernel (zuf) the app thread is awaken with the results, and the
+     ZT thread goes back to sleep waiting a new operation.
+     
+   ZT rules:
+       A ZT thread must not return back to Kernel. One exception is locks
+   if needed it might sleep waiting for a lock. In which case we will see that
+   the same CPU channel is reentered via another application and/or thread.
+   But now that CPU channel is taken.  What we do is we utilize a few channels
+   (ZTs) per core and the threads may grab another channel. But this only
+   postpones the problem on a busy contended system, all such channels will be
+   consumed. If all channels are taken the application thread is put on a busy
+   scheduling wait until a channel can be grabbed.
+   Therefor Server must not sleep on a ZT. If it needs such a sleeping operation
+   it will return -EAGAIN to zuf. The app is kept sleeping the operation is put
+   on an asynchronous Q and the ZT freed for foreground operation. At some point
+   when the server completes the delayed operation it will complete notify
+   the Kernel with a special async cookie. And the app will be awakened.
+   (Here too we utilize pre allocated asyc channels and vmas. If all channels
+    are busy, application is kept sleeping waiting its free slot turn)
+
+4. On umount the operation is reversed and all resources are torn down.
+5. In case of an application or Server crash, all resources are Associated
+   with files, on file_release these resources are caught and freed.
+
+Objects and life-time
+~~~~~~~~~~~~~~~~~~~~~
+

Documentation/filesystems/zufs.txt line 255

Quoted 10 lines
Documentation/filesystems/zufs.txt new file +10 -0
+Each Kernel object type has an assosiated zus Server object type who's life
+time is governed by the life-time of the Kernel object. Therefor the Server's
+job is easy because it need not establish any object caches / hashes and so on.
+
+Inside zus all objects are allocated by the zusFS plugin. So in turn it can
+allocate a bigger space for its own private data and access it via the
+container_off() coding pattern. So when I say below a zus-object I mean both
+zus public part + zusFS private part of the same object.
+
+All operations return a UM pointer that are OPEC the the Kernel code, they

-ETOOMANYNLA

2LA: UM
4LA: OPEC

Quoted 87 lines
Documentation/filesystems/zufs.txt new file +86 -0
+are just a cookie which is returned back to zus, when needed.
+At times when we want the Kernel to have direct access to a zus object like
+zus_inode, along with the cookie we also return a dpp_t, with a defined structure.
+
+Kernel object 			| zus object 		| Kernel access (via dpp_t)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+zuf_fs_type
+	file_system_type	| zus_fs_info		| no
+
+zuf_sb_info
+	super_block		| zus_sb_info		| no
+	
+zuf_inode_info			|			|
+	vfs_inode		| zus_inode_info	| no
+	zus_inode *		| 	zus_inode *	| yes
+	synlink *		|	char-array	| yes
+	xattr**			|	zus_xattr	| yes
+
+When a Kernel object's time is to die, a final call to zus is
+dispatched so the associated object can also be freed. Which means
+that on memory pressure when object caches are evicted also the zus
+memory resources are freed.
+
+
+How to use zufs:
+~~~~~~~~~~~~~~~~
+
+The most updated documentation of how to use the latest code bases
+is the script (set of scripts) at fs/do-zu/zudo on the zus git tree
+
+We the developers at Netapp use this script to mount and test our
+latest code. So any new Secret will be found in these scripts. Please
+read them as the ultimate source of how to operate things.
+
+TODO: We are looking for exports in system-d and udev to properly
+integrate these tools into a destro.
+
+We assume you cloned these git trees:
+[]$ mkdir zufs; cd zufs
+[]$ git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+[]$ git clone https://github.com/NetApp/zufs-zuf -b zus-github
+
+This will create the following trees
+zufs/zus - Source code for Server
+zufs/zuf - Linux Kernel source tree to compile and install on your machine
+
+Also specifically:
+zufs/zus/fs/do-zu/zudo - script Documenting how to run things
+
+[]$ cd zuf
+
+First time
+[] ../zus/fs/do-zu/zudo
+this will create a file:
+	../zus/fs/do-zu/zu.conf
+
+Edit this file for your environment. Devices, mount-point and so on.
+On first run an example file will be created for you. Fill in the
+blanks. Most params can stay as is in most cases
+
+Now lest start running:
+
+[1]$ ../zus/fs/do-zu/zudo mkfs
+This will run the proper mkfs command selected at zu.conf file
+with the proper devices.
+
+[2]$ ../zus/fs/do-zu/zudo zuf-insmod
+This loads the zuf.ko module
+
+[3]$ ../zus/fs/do-zu/zudo zuf-root
+This mounts the zuf-root FS above on /sys/fs/zuf (automatically created above)
+
+[4]$ ../zus/fs/do-zu/zudo zus-up
+This runs the zus daemon in the background
+
+[5]$ ../zus/fs/do-zu/zudo mount
+This mount the mkfs FS above on the specified dir in zu.conf
+
+To run all the 5 commands above at once do:
+[]$ ../zus/fs/do-zu/zudo up
+
+To undo all the above in reverse order do:
+[]$ ../zus/fs/do-zu/zudo down
+
+And the most magic command is:
+[]$ ../zus/fs/do-zu/zudo again

Documentation/filesystems/zufs.txt line 351

Quoted 10 lines
Documentation/filesystems/zufs.txt new file +10 -0
+Will do a "down", then update-mods, then "up"
+(update-mods is a special script to copy the latest compiled binaries)
+
+Now you are ready for some:
+[]$ ../zus/fs/do-zu/zudo xfstest
+xfstests is assumed to be installed in the regular /opt/xfstests dir
+
+Again please see inside the scripts what each command does
+these scripts are the ultimate Documentation, do not believe
+anything I'm saying here. (Because it is outdated by now)

thanks,

Signature · 1 line
~Randy
#29 lore raw

Re: [RFC 3/7] zuf: Preliminary Documentation

Quoted 67 lines · Randy Dunlap

Hi,

Just a few questions. Very little editing. :)

Quoted 61 lines · Boaz Harrosh

Adding Documentation/filesystems/zufs.txt

---
 Documentation/filesystems/zufs.txt | 351 +++++++++++++++++++++++++++++++++++++
 1 file changed, 351 insertions(+)
 create mode 100644 Documentation/filesystems/zufs.txt
Documentation/filesystems/zufs.txt new file +46 -0
new file mode 100644
index 0000000..779f14b
@@ -0,0 +1,351 @@
+ZUFS - Zero-copy User-mode FileSystem
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Trees:
+	git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+	git clone https://github.com/NetApp/zufs-zus -b zus-github
+
+patches, comments, questions, requests to:
+	[email protected]
+
+Introduction:
+~~~~~~~~~~~~~
+
+ZUFS - stands for Zero-copy User-mode FS
+▪ It is geared towards true zero copy end to end of both data and meta data.
+▪ It is geared towards very *low latency*, very high CPU locality, lock-less
+  parallelism.
+▪ Synchronous operations
+▪ Numa awareness
+
+ ZUFS is a, from scratch, implementation of a filesystem-in-user-space, which
+tries to address the above goals. It is aimed for pmem based FSs. But can easily
+support any other type of FSs that can utilize x10 latency and parallelism
+improvements.
+
+Glossary and names:
+~~~~~~~~~~~~~~~~~~~
+
+ZUF - Zero-copy User-mode Feeder
+  zuf.ko is the Kernel VFS component. Its job is to interface with the Kernel
+  VFS and dispatch commands to a User-mode application Server.
+  Uptodate code is found at:
+	git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+
+ZUS - Zero-copy User-mode Server
+  zufs utilizes a User-mode server application. That takes care of the detailed
+  communication protocol and correctness with the Kernel.
+  In turn it utilizes many zusFS Filesystem plugins to implement the actual
+  on disc Filesystem.
+  Uptodate code is found at:
+	git clone https://github.com/NetApp/zufs-zus -b zus-github
+
+zusFS - FS plugins
+  These are .so loadable modules that implement one or more Filesystem-types
+  (-t xyz).
+  The zus server communicates with the plugin via a set of function vectors
Quoted 9 lines

Documentation/filesystems/zufs.txt line 52

Quoted 6 lines
Documentation/filesystems/zufs.txt new file +6 -0
+  for the different operations. And establishes communication via defined
+  structures.
+
+Filesystem-type:
+  At startup zus registers with the Kernel one or more Filesystem-type(s)
+  Associated with the type is a 4 letter type-name (-t fstn) different

(?) is a unique 4-letter type-name

FStype-name I guess. this field in the Kernel is called file_system_type.name and is just a pointer so I guess it can be any length. I have space for 15 characters in this API.
But I agree that the sentence above is clear as mud I will reword it

Quoted 9 lines
Quoted 9 lines
Documentation/filesystems/zufs.txt new file +9 -0
+  info about the fs, like a magic number and so on.
+  One Server can support many FS-types, in turn each FS-type can mount
+  multiple super-blocks, each supporting multiple devices.
+
+Device-Table (DT) - A zufs FS can support multiple devices
+  ZUF in Kernel may receive, like any mount command a block-device or none.
+  For the former if the specified FS-types states so in a special field.
+  The mount will look for a Device table. A list of devices in a specific
+  order sitting at some offset on the block-device. The system will then
Quoted 9 lines

Documentation/filesystems/zufs.txt line 67

Quoted 6 lines
Documentation/filesystems/zufs.txt new file +6 -0
+  proceed to open and own all these devices and associate them to the mounting
+  super-block.
+  If FS-type specifies a -1 at DT_offset then there is no device table
+  and a DT of a single device is created. (If we have no devices, none
+  is specified than we operate without any block devices. (Mount options give
+  some indication of the storage information)

missing one ')'

yep

Quoted 6 lines

Documentation/filesystems/zufs.txt line 70

Quoted 3 lines
Documentation/filesystems/zufs.txt new file +3 -0
+  The device table has special consideration for pmem devices and will
+  present the all linear array of devices to zus, as one flat mmap space.
+  Alternatively all none pmem devices are also provided an interface

all known (?)

No this is none-pmem .i.e other devices that are regular bdev(s) and do not give the special pmem interface. Will fix

Quoted 14 lines
Quoted 14 lines
Documentation/filesystems/zufs.txt new file +14 -0
+  with facility of data movement from pmem to a slower device.
+  A detailed NUMA info is exported to the Server for maximum utilization.
+
+pmem:
+  Multiple pmem devices are presented to the server as a single
+  linear file mmap. Something like /dev/dax. But it is strictly
+  available only to that specific super-block that owns it.
+
+dpp_t - Dual port pointer type
+  At some points in the protocol there are objects that return from zus
+  (The Server) to the Kernel via a dpp_t. This is a special kind of pointer
+  It is actually an offset 8 bytes aligned with the 3 low bits specifying
+  a pool code: [offset = dpp_t & ~0x7] [pool = dpp_t & 0x7]
+  pool == 0 means the offset is in pmem who's management is by zuf and
Quoted 9 lines

Documentation/filesystems/zufs.txt line 89

Quoted 5 lines
Documentation/filesystems/zufs.txt new file +5 -0
+  a full easy access is provided for zus.
+
+  pool != 0 Is a pre-established tempfs file (up to 6 such files) where
+  the zus has an mmap on the file and the Kernel can access that data
+  via an offset into the file.

so non-zero pool [dpp_t & 0x7] can be a value of 1 - 7, and above says up to 6 such tempfs files. What is the other pool value used for?

Yes my bad. Thanks

Quoted 5 lines
Quoted 5 lines
Documentation/filesystems/zufs.txt new file +5 -0
+  All dpp_t objects life time rules are strictly defined.
+  Mainly the primary use of dpp_t is the on-pmem inode structure. Both
+  zus and zuf can access and change this structure. On any modification
+  the zus is called so to be notified of any changes, persistence.
+  More such objects are: Symlinks, xattrs, mmap-data-blocks etc...
Quoted 9 lines

Documentation/filesystems/zufs.txt line 100

Quoted 6 lines
Documentation/filesystems/zufs.txt new file +6 -0
+
+Relay-wait-object:
+  communication between Kernel and server are done via zus-threads that
+  sleep in Kernel (inside an IOCTL) and wait for commands. Once received
+  the IOCTL returns operation id executed and the return info is returned via
+  a new IOCTL call, which then waits for the next operation.

Does that say 2 IOCTLs per command? One to start it and one to fetch return info?

Not really, I guess I did not explain well. It is one IOCTL per command + the very very first one. It is a cyclic system. The IOCTL that returns results is then sleeping again to wait for a new command.

Quoted 1 line

From the perspective of the user-mode code this is a single IORW (both write+read)

type IOCTL that sends some information - command-results - and receives some information - A new command.

How to explain this better.

Quoted 32 lines
Quoted 32 lines
Documentation/filesystems/zufs.txt new file +32 -0
+  To wake up the sleeping thread we use a Relay-wait-object. Currently
+  it is two waitqueue_head(s) back to back.
+  In future we should investigate the use of that special binder object
+  that releases its thread time slice to the other thread without going through
+  the scheduler.
+
+ZT-threads-array:
+  The novelty of the zufs is the ZT-threads system. One thread or more is
+  pre-created for each active core in the system.
+  ▪ The thread is AFFINITY set for that single core only.
+  ▪ Special communication file per ZT (O_TMPFILE + IOCTL_ZUFS_INIT)
+    At initialization the ZT thread communicates through a ZT_INIT ioctl
+    and registers as the handler of that core (Channel)
+  ▪ ZT-vma - Mmap 4M vma zero copy communication area per ZT
+    Pre allocated vma is created into which will be mapped the application
+    or Kernel buffers for the current operation.
+  ▪ IOCTL_ZU_WAIT_OPT – threads sleeps in Kernel waiting for an operation
+    via the IOCTL_ZU_WAIT_OPT call. supplying a 4k communication buffer
+
+  ▪ On an operation dispatch current CPU's ZT is selected, app pages mapped
+    into the ZT-vma. Server thread released with an operation to execute.
+  ▪ After execution, ZT returns to kernel (IOCTL_ZU_WAIT_OPT), app is released,
+    Server wait for new operation on that CPU.
+
+ZUS-mount-thread:
+  The system utilizes a single mount thread. (This thread is not affinity to any
+  core).
+  ▪ It will first Register all FS-types supported by this Server (By calling
+    all zusFS plugins to register their supported types). Once done
+  ▪ As above the thread sleeps in Kernel via the IOCTL_ZU_MOUNT call.
+  ▪ When the Kernel receives a mount request (vfs calles the fs_type->mount opt)
+    a mount is dispatched back to zus.
Quoted 9 lines

Documentation/filesystems/zufs.txt line 138

Quoted 6 lines
Documentation/filesystems/zufs.txt new file +6 -0
+  ▪ NOTE: That only on very first mount the above ZT-threads-array is created
+    the same array is then used for all super-blocks in the system
+  ▪ As part of the mount command in the context of this same mount-thread
+    a call to IOCTL_ZU_GRAB_PMEM will establish an interface to the pmem
+    Associated with this super_block
+  ▪ On return (like above a new call to IOCTL_ZU_MOUNT will return info of the

missing ')' somewhere.

Yes

Quoted 36 lines
Quoted 36 lines
Documentation/filesystems/zufs.txt new file +36 -0
+    mount before sleeping in kernel waiting for a new dispatch. All SB info
+    is provided to zuf, including the root inode info. Kernel then proceeds
+    to complete the mount call.
+  ▪ NOTE that since there is a single mount thread all FS-registration
+    super_block and pmem management are lockless.
+  
+Philosophy of operations:
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. [zuf-root]
+
+On module load  (zuf.ko) A special pseudo FS is mounted on /sys/fs/zuf. This is
+called zuf-root.
+The zuf-root has no visible files. All communication is done via special-files.
+special-files are open(O_TMPFILE) and establish a special role via an
+IOCTL.
+All communications with the server are done via the zuf-root. Each root owns
+many FS-types and each FS-type owns many super-blocks of this type. All Sharing
+the same communication channels.
+Since all FS-type Servers live in the same zus application address space, at
+times. If the administrator wants to separate between different servers, he/she
+can mount a new zuf-root and point a new server instance on that new mount,
+registering other FS-types on that other instance. The all communication array
+will then be duplicated as well.
+(Otherwise pointing a new server instance on a busy root will return an error)
+
+2. [zus server start]
+  ▪ On load all configured zusFS plugins are loaded.
+  ▪ The Server starts by starting a single mount thread.
+  ▪ It than proceeds to register with Kernel all FS-types it will support.
+    (This is done on the single mount thread, so all FS-registration and
+     mount/umount operate in a single thread and therefor need not any locks)
+  ▪ Sleeping in the Kernel on a special-file of that zuf-root. waiting for a mount
+    command.
+
+3. [mount -t xyz]
Quoted 9 lines

Documentation/filesystems/zufs.txt line 180

Quoted 6 lines
Documentation/filesystems/zufs.txt new file +6 -0
+  [In Kernel]
+  ▪ If xyz was registered above as part of the Server startup. the regular
+    mount command will come to the zuf module with a zuf_mount() call. with
+    the xyz-FS-info. In turn this points to a zuf-root.
+  ▪ Code than proceed to load a device-table of devices as  specified above.
+    It then establishes an md object with a specific pmem_id.
                              md ??

Thanks yes I should explain better MD stands for multi_devices which is the name of the structure. I will put some definitions somewhere

Quoted 78 lines
Quoted 65 lines
Documentation/filesystems/zufs.txt new file +65 -0
+  ▪ It proceeds to call mount_bdev. Always with the same main-device
+    thous fully sporting automatic bind mounts. Even if different
+    devices are given to the mount command.
+  ▪ In zuf_fill_super it will then dispatch (awaken) the mount thread
+    specifying two parameters. One the FS-type to mount, and then
+    the pmem_id Associated with this super_block.
+
+  [In zus]
+  ▪ A zus_super_block_info is allocated.
+  ▪ zus calls PMEM_GRAB(pmem_id) to establish a direct mapping to its
+    pmem devices. On return we have full access to our PMEM
+
+  ▪ ZT-threads-array
+    If this is the first mount the all ZT-threads-array is created and
+    established. The mount thread will wait until all zt-threads finished
+    initialization and ready to rock.
+  ▪ Root-zus_inode is loaded and is returned to kernel
+  ▪ More info about the mount like block sizes and so on are returned to kernel.
+
+  [In Kernel]
+   The zuf_fill_super is finalized vectors established and we have a new
+   super_block ready for operations.
+
+4. An FS operation like create or WRITE/READ and so on arrives from application
+   via VFS. Eventually an Operation is dispatched to zus:
+   ▪ A special per-operation descriptor is filled up with all parameters.
+   ▪ A current CPU channel is grabbed. the operation descriptor is put on
+     that channel (ZT). Including get_user_pages or Kernel-pages associated
+     with this OPT.
+   ▪ The ZT is awaken, app thread put to sleep.
+   ▪ In ZT context pages are mapped to that ZT-vma. This is so we are sure
+     the map is only on a single core. And no other core's TLB is affected.
+     (This here is the all performance secret)
+   ▪ ZT thread is returned to user-space.
+   ▪ In ZT context the zus Server calls the appropriate zusFS->operation
+     vector. Output params filled.
+   ▪ zus calls again with an IOCTL_ZU_WAIT_OPT with the same descriptor
+     to return the requested info.
+   ▪ At Kernel (zuf) the app thread is awaken with the results, and the
+     ZT thread goes back to sleep waiting a new operation.
+     
+   ZT rules:
+       A ZT thread must not return back to Kernel. One exception is locks
+   if needed it might sleep waiting for a lock. In which case we will see that
+   the same CPU channel is reentered via another application and/or thread.
+   But now that CPU channel is taken.  What we do is we utilize a few channels
+   (ZTs) per core and the threads may grab another channel. But this only
+   postpones the problem on a busy contended system, all such channels will be
+   consumed. If all channels are taken the application thread is put on a busy
+   scheduling wait until a channel can be grabbed.
+   Therefor Server must not sleep on a ZT. If it needs such a sleeping operation
+   it will return -EAGAIN to zuf. The app is kept sleeping the operation is put
+   on an asynchronous Q and the ZT freed for foreground operation. At some point
+   when the server completes the delayed operation it will complete notify
+   the Kernel with a special async cookie. And the app will be awakened.
+   (Here too we utilize pre allocated asyc channels and vmas. If all channels
+    are busy, application is kept sleeping waiting its free slot turn)
+
+4. On umount the operation is reversed and all resources are torn down.
+5. In case of an application or Server crash, all resources are Associated
+   with files, on file_release these resources are caught and freed.
+
+Objects and life-time
+~~~~~~~~~~~~~~~~~~~~~
+

Documentation/filesystems/zufs.txt line 255

Quoted 10 lines
Documentation/filesystems/zufs.txt new file +10 -0
+Each Kernel object type has an assosiated zus Server object type who's life
+time is governed by the life-time of the Kernel object. Therefor the Server's
+job is easy because it need not establish any object caches / hashes and so on.
+
+Inside zus all objects are allocated by the zusFS plugin. So in turn it can
+allocate a bigger space for its own private data and access it via the
+container_off() coding pattern. So when I say below a zus-object I mean both
+zus public part + zusFS private part of the same object.
+
+All operations return a UM pointer that are OPEC the the Kernel code, they

-ETOOMANYNLA

Quoted 2 lines

2LA: UM
4LA: OPEC

Yes thanks I will change

Quoted 91 lines
Quoted 91 lines
Documentation/filesystems/zufs.txt new file +91 -0
+are just a cookie which is returned back to zus, when needed.
+At times when we want the Kernel to have direct access to a zus object like
+zus_inode, along with the cookie we also return a dpp_t, with a defined structure.
+
+Kernel object 			| zus object 		| Kernel access (via dpp_t)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+zuf_fs_type
+	file_system_type	| zus_fs_info		| no
+
+zuf_sb_info
+	super_block		| zus_sb_info		| no
+	
+zuf_inode_info			|			|
+	vfs_inode		| zus_inode_info	| no
+	zus_inode *		| 	zus_inode *	| yes
+	synlink *		|	char-array	| yes
+	xattr**			|	zus_xattr	| yes
+
+When a Kernel object's time is to die, a final call to zus is
+dispatched so the associated object can also be freed. Which means
+that on memory pressure when object caches are evicted also the zus
+memory resources are freed.
+
+
+How to use zufs:
+~~~~~~~~~~~~~~~~
+
+The most updated documentation of how to use the latest code bases
+is the script (set of scripts) at fs/do-zu/zudo on the zus git tree
+
+We the developers at Netapp use this script to mount and test our
+latest code. So any new Secret will be found in these scripts. Please
+read them as the ultimate source of how to operate things.
+
+TODO: We are looking for exports in system-d and udev to properly
+integrate these tools into a destro.
+
+We assume you cloned these git trees:
+[]$ mkdir zufs; cd zufs
+[]$ git clone https://github.com/NetApp/zufs-zuf -b zuf-upstream
+[]$ git clone https://github.com/NetApp/zufs-zuf -b zus-github
+
+This will create the following trees
+zufs/zus - Source code for Server
+zufs/zuf - Linux Kernel source tree to compile and install on your machine
+
+Also specifically:
+zufs/zus/fs/do-zu/zudo - script Documenting how to run things
+
+[]$ cd zuf
+
+First time
+[] ../zus/fs/do-zu/zudo
+this will create a file:
+	../zus/fs/do-zu/zu.conf
+
+Edit this file for your environment. Devices, mount-point and so on.
+On first run an example file will be created for you. Fill in the
+blanks. Most params can stay as is in most cases
+
+Now lest start running:
+
+[1]$ ../zus/fs/do-zu/zudo mkfs
+This will run the proper mkfs command selected at zu.conf file
+with the proper devices.
+
+[2]$ ../zus/fs/do-zu/zudo zuf-insmod
+This loads the zuf.ko module
+
+[3]$ ../zus/fs/do-zu/zudo zuf-root
+This mounts the zuf-root FS above on /sys/fs/zuf (automatically created above)
+
+[4]$ ../zus/fs/do-zu/zudo zus-up
+This runs the zus daemon in the background
+
+[5]$ ../zus/fs/do-zu/zudo mount
+This mount the mkfs FS above on the specified dir in zu.conf
+
+To run all the 5 commands above at once do:
+[]$ ../zus/fs/do-zu/zudo up
+
+To undo all the above in reverse order do:
+[]$ ../zus/fs/do-zu/zudo down
+
+And the most magic command is:
+[]$ ../zus/fs/do-zu/zudo again
+Will do a "down", then update-mods, then "up"
+(update-mods is a special script to copy the latest compiled binaries)
+
+Now you are ready for some:
+[]$ ../zus/fs/do-zu/zudo xfstest
Quoted 9 lines

Documentation/filesystems/zufs.txt line 351

Quoted 6 lines
Documentation/filesystems/zufs.txt new file +5 -0
+xfstests is assumed to be installed in the regular /opt/xfstests dir
+
+Again please see inside the scripts what each command does
+these scripts are the ultimate Documentation, do not believe
+anything I'm saying here. (Because it is outdated by now)

thanks,

Thank you Randy for all the input.
I always have such hard time with the Documentation and very low confidence in it. As I was slaving over this I hoped you'd have a bit of time to look it over.

Will fix
Boaz

reviewer
#30 lore raw

Re: [RFC 3/7] zuf: Preliminary Documentation

Quoted 9 lines · Boaz Harrosh
Quoted 3 lines · Randy Dunlap

Hi,

Just a few questions. Very little editing. :)

Thank you Randy for all the input.
I always have such hard time with the Documentation and very low confidence in it. As I was slaving over this I hoped you'd have a bit of time to look it over.

I will make another pass over the next version to fix lots of typos, grammar, spelling, etc.

Signature · 1 line
~Randy
#32 lore raw

RFC 4/7 zuf: zuf-rootfs && zuf-core

zuf-core established the communication channels with the zus UM Server.

zuf-root is a psuedo FS that the zus communicates through, registers new file-systems. receives new mount requests.

In this patch we have the bring up of that special FS, and the core communication mechanics. Which is the Novelty of this code submission.

The zuf-rootfs (-t zuf) is usually by default mounted on /sys/fs/zuf. If an admin wants to run more server applications (Note that each server application supports many types of FSs) He/she can mount a second instance of -t zuf and point the new Server to it.

(Otherwise a second instance attempting to communicate with a
busy zuf will fail)

TODO: How to trigger a first mount on module_load. Currently admin needs to manually "mount -t zuf none /sys/fs/zuf"

---
 fs/zuf/Makefile   |   4 +
 fs/zuf/_extern.h  |  47 +++++
 fs/zuf/_pr.h      |  53 ++++++
 fs/zuf/relay.h    |  86 +++++++++
 fs/zuf/super.c    |  21 +++
 fs/zuf/zuf-core.c | 517 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/zuf/zuf-root.c | 330 ++++++++++++++++++++++++++++++++++
 fs/zuf/zuf.h      |  90 ++++++++++
 fs/zuf/zus_api.h  | 108 ++++++++++++
 9 files changed, 1256 insertions(+)
 create mode 100644 fs/zuf/_extern.h
 create mode 100644 fs/zuf/_pr.h
 create mode 100644 fs/zuf/relay.h
 create mode 100644 fs/zuf/super.c
 create mode 100644 fs/zuf/zuf-core.c
 create mode 100644 fs/zuf/zuf-root.c
 create mode 100644 fs/zuf/zuf.h
fs/zuf/Makefile +4 -0
index 7e4e51f..d00940c 100644
@@ -10,5 +10,9 @@
 
 obj-$(CONFIG_ZUF) += zuf.o
 
+# ZUF core
+zuf-y += zuf-core.o zuf-root.o
+
 # Main FS
+zuf-y += super.o
 zuf-y += module.o
fs/zuf/_extern.h new file +47 -0

47 lines not on this page · raw message

fs/zuf/_pr.h new file +53 -0

53 lines not on this page · raw message

fs/zuf/relay.h new file +86 -0

86 lines not on this page · raw message

fs/zuf/super.c new file +21 -0
new file mode 100644
index 0000000..6e176a5
@@ -0,0 +1,21 @@
+/*
+ * BRIEF DESCRIPTION
+ *
+ * Super block operations.
+ *
+ * Copyright (c) 2018 NetApp Inc. All rights reserved.
+ *
+ * ZUFS-License: GPL-2.0 OR BSD-3-Clause. See module.c for LICENSE details.
+ *
+ * Authors:
+ *	Boaz Harrosh <[email protected]>
+ *	Sagi Manole <[email protected]>"
+ */
+
+#include "zuf.h"
+
+struct dentry *zuf_mount(struct file_system_type *fs_type, int flags,
+			 const char *dev_name, void *data)
+{
+	return ERR_PTR(-ENOTSUPP);
+}
fs/zuf/zuf-core.c new file +517 -0

517 lines not on this page · raw message

fs/zuf/zuf-root.c new file +330 -0

330 lines not on this page · raw message

fs/zuf/zuf.h new file +90 -0

90 lines not on this page · raw message

fs/zuf/zus_api.h +108 -0

112 lines not on this page · raw message

Signature · 1 line
2.5.5
#33 lore raw

Re: [RFC 4/7] zuf: zuf-rootfs && zuf-core

Quoted 24 lines · Boaz Harrosh

zuf-core established the communication channels with the zus UM Server.

zuf-root is a psuedo FS that the zus communicates through, registers new file-systems. receives new mount requests.

In this patch we have the bring up of that special FS, and the core communication mechanics. Which is the Novelty of this code submission.

The zuf-rootfs (-t zuf) is usually by default mounted on /sys/fs/zuf. If an admin wants to run more server applications (Note that each server application supports many types of FSs) He/she can mount a second instance of -t zuf and point the new Server to it.

(Otherwise a second instance attempting to communicate with a
busy zuf will fail)

TODO: How to trigger a first mount on module_load. Currently admin needs to manually "mount -t zuf none /sys/fs/zuf"

<snip>

Review

Quoted 2 lines
(quoted diff) +2 -0
+	while (!relay_is_fss_waiting(&zt->relay)) {
+		mb();

Not something for this early in the development cycle but something to keep in mind:

Always document all assumptions re. memory barriers usage + intended pairing scenario otherwise it's very hard to reason whether this is correct or not. In fact barriers without comments are considered broken.

Quoted 16 lines
fs/zuf/zuf-core.c new file +16 -0
+		if (unlikely(!zt->file))
+			return -EIO;
+		zuf_dbg_err("[%d] can this be\n", cpu);
+		/* FIXME: Do something much smarter */
+		msleep(10);
+		mb();
+	}
+
+	zt->next_opt = hdr;
+	zt->pages = pages;
+	zt->nump = nump;
+
+	relay_fss_wakeup_app_wait(&zt->relay, NULL);
+
+	return zt->file ? hdr->err : -EIO;
+}

<snip>

#34 lore raw

Re: [RFC 4/7] zuf: zuf-rootfs && zuf-core

Quoted 29 lines · Nikolay Borisov
Quoted 24 lines · Boaz Harrosh

zuf-core established the communication channels with the zus UM Server.

zuf-root is a psuedo FS that the zus communicates through, registers new file-systems. receives new mount requests.

In this patch we have the bring up of that special FS, and the core communication mechanics. Which is the Novelty of this code submission.

The zuf-rootfs (-t zuf) is usually by default mounted on /sys/fs/zuf. If an admin wants to run more server applications (Note that each server application supports many types of FSs) He/she can mount a second instance of -t zuf and point the new Server to it.

(Otherwise a second instance attempting to communicate with a
busy zuf will fail)

TODO: How to trigger a first mount on module_load. Currently admin needs to manually "mount -t zuf none /sys/fs/zuf"

<snip>

Quoted 9 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	while (!relay_is_fss_waiting(&zt->relay)) {
+		mb();

Not something for this early in the development cycle but something to keep in mind:

Always document all assumptions re. memory barriers usage + intended pairing scenario otherwise it's very hard to reason whether this is correct or not. In fact barriers without comments are considered broken.

Yes. I totally agree. I love it how you commented on the ugliest piece of code in all of this.

This is BTW totally very wrong and the mb is wrong and is a bandade over the wrong kind of hurt. Because in theory coming out of sleep I might be scheduled on another core and I should then pick up a new zt instead of syncing with the now wrong one.

for a POC it was fine, I get here maybe 0.17% of the times but I will totally need to put some thought and love into this problem.

I kind of want to move from that Relay object to what Binder is using and hope to shave off some more latency. So I'll see if I will fix this or move to new code

Thank you for looking, Yes this is a bad contraption Boaz

Quoted 19 lines
Quoted 16 lines
fs/zuf/zuf-core.c new file +16 -0
+		if (unlikely(!zt->file))
+			return -EIO;
+		zuf_dbg_err("[%d] can this be\n", cpu);
+		/* FIXME: Do something much smarter */
+		msleep(10);
+		mb();
+	}
+
+	zt->next_opt = hdr;
+	zt->pages = pages;
+	zt->nump = nump;
+
+	relay_fss_wakeup_app_wait(&zt->relay, NULL);
+
+	return zt->file ? hdr->err : -EIO;
+}

<snip>

#36 lore raw

RFC 5/7 zus: Devices && mounting

In this patch we already establish a mounted filesystem.

There are three mode of operations:
* mount with out a device (mount -t FOO none /somepath)

* A single device - The FS stated register_fs_info->dt_offset==-1 No checks are made by Kernel, the single bdev is registered with Kernel's mount_bdev. It is up to the zusFS to check validity

* Multy devices - The FS stated register_fs_info->dt_offset==X This mode is the main of this patch.
A single device is given on the mount command line. At register_fs_info->dt_offset of this device we look for a zufs_dev_table structure. After all the checks we look there at the device list and open all devices. Any one of the devices may be given on command line. But they will always be opened in DT(Device Table) order. The Device table has the notion of two types of bdevs:
T1 devices - are pmem devices capable of direct_access
T2 devices - are none direct_access devices

All t1 devices are presented as one linear array. in DT order In t1.c we mmap this space for the server to directly access pmem. (In the proper persistent way)

[We do not support any direct_access device, we only support
pmem(s) where the all device can be addressed by a single physical/virtual address. This is checked before mount]

The T2 devices are also grabbed and owned by the super_block A later API will enable the Server to write or transfer buffers from T1 to T2 in a very efficient manner. Also presented as a single linear array in DT order.

Both kind of devices are NUMA aware and the NUMA info is presented to the zusFS for optimal allocation and access.

So these are the steps for mounting a zufs Filesystem:

* All devices (Single or DT) are opened and established in an md object. This md-object is given an pmem-id

* mount_bdev is called with the main (first) device, in turn fill_supper is called.

* fill_supper despatches a mount_operation(register_fs_info) to the server with the pmem-id of the md-object above.

* The Server at the zus mount routine. Will first thing do a GRAB_PMEM(pmem-id) ioctl call to establish a special filehandle through which it will have full access to the all of its pmem space. With that it will call the zusFS to continue to inspect the content of the pmem and mount the FS.

* On return from mount the zusFS returns the root inode info

* fill_supper continues to create a root vfs-inode and returns successfully.

* We now have a mounted super_block, with corresponding super_block objects in the Server.

---
 fs/zuf/Makefile   |   5 +-
 fs/zuf/_extern.h  |  12 +
 fs/zuf/inode.c    |  45 ++++
 fs/zuf/md.c       | 695 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/zuf/md.h       | 188 +++++++++++++++
 fs/zuf/super.c    | 605 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/zuf/t1.c       | 114 +++++++++
 fs/zuf/t2.c       | 348 +++++++++++++++++++++++++++
 fs/zuf/t2.h       |  67 ++++++
 fs/zuf/zuf-core.c |  68 +++++-
 fs/zuf/zuf-root.c |   9 +-
 fs/zuf/zuf.h      | 220 +++++++++++++++++
 fs/zuf/zus_api.h  | 177 ++++++++++++++
 13 files changed, 2549 insertions(+), 4 deletions(-)
 create mode 100644 fs/zuf/inode.c
 create mode 100644 fs/zuf/md.c
 create mode 100644 fs/zuf/md.h
 create mode 100644 fs/zuf/t1.c
 create mode 100644 fs/zuf/t2.c
 create mode 100644 fs/zuf/t2.h
fs/zuf/Makefile +4 -1
index d00940c..94ce80b 100644
@@ -10,9 +10,12 @@
 
 obj-$(CONFIG_ZUF) += zuf.o
 
+# Infrastructure
+zuf-y += md.o t2.o t1.o
+
 # ZUF core
 zuf-y += zuf-core.o zuf-root.o
 
 # Main FS
-zuf-y += super.o
+zuf-y += super.o inode.o
 zuf-y += module.o
fs/zuf/_extern.h +12 -0
index e490043..0543fd8 100644
@@ -19,7 +19,16 @@
  * extern functions declarations
  */
 
+/* inode.c */
+struct inode *zuf_iget(struct super_block *sb, struct zus_inode_info *zus_ii,
+		       zu_dpp_t _zi, bool *exist);
+void zuf_evict_inode(struct inode *inode);
+int zuf_write_inode(struct inode *inode, struct writeback_control *wbc);
+int zuf_isync(struct inode *inode, loff_t start, loff_t end, int datasync);
+
 /* super.c */
+int zuf_init_inodecache(void);
+void zuf_destroy_inodecache(void);
 struct dentry *zuf_mount(struct file_system_type *fs_type, int flags,
 			 const char *dev_name, void *data);
 
@@ -44,4 +53,7 @@ void zufs_mounter_release(struct file *filp);
 /* zuf-root.c */
 int zuf_register_fs(struct super_block *sb, struct zufs_ioc_register_fs *rfs);
 
+/* t1.c */
+int zuf_pmem_mmap(struct file *file, struct vm_area_struct *vma);
+
 #endif	/*ndef __ZUF_EXTERN_H__*/
fs/zuf/inode.c new file +45 -0

45 lines not on this page · raw message

fs/zuf/md.c new file +695 -0

695 lines not on this page · raw message

fs/zuf/md.h new file +188 -0

188 lines not on this page · raw message

fs/zuf/super.c +604 -1

614 lines not on this page · raw message

fs/zuf/t1.c new file +114 -0

114 lines not on this page · raw message

fs/zuf/t2.c new file +348 -0

348 lines not on this page · raw message

fs/zuf/t2.h new file +67 -0

67 lines not on this page · raw message

fs/zuf/zuf-core.c +67 -1

86 lines not on this page · raw message

fs/zuf/zuf-root.c +8 -1
index 8102d3a..ebb44c9 100644
@@ -117,6 +117,8 @@ int zufr_mmap(struct file *file, struct vm_area_struct *vma)
 	switch (zsf->type) {
 	case zlfs_e_zt:
 		return zuf_zt_mmap(file, vma);
+	case zlfs_e_pmem:
+		return zuf_pmem_mmap(file, vma);
 	default:
 		zuf_err("type=%d\n", zsf->type);
 		return -ENOTTY;
@@ -300,7 +302,10 @@ static struct kset *zufr_kset;
 
 int __init zuf_root_init(void)
 {
-	int err;
+	int err = zuf_init_inodecache();
+
+	if (unlikely(err))
+		return err;
 
 	zufr_kset = kset_create_and_add("zuf", NULL, fs_kobj);
 	if (!zufr_kset) {
@@ -317,6 +322,7 @@ int __init zuf_root_init(void)
 un_kset:
 	kset_unregister(zufr_kset);
 un_inodecache:
+	zuf_destroy_inodecache();
 	return err;
 }
 
@@ -324,6 +330,7 @@ void __exit zuf_root_exit(void)
 {
 	unregister_filesystem(&zufr_type);
 	kset_unregister(zufr_kset);
+	zuf_destroy_inodecache();
 }
 
 module_init(zuf_root_init)
fs/zuf/zuf.h +220 -0

239 lines not on this page · raw message

fs/zuf/zus_api.h +177 -0

204 lines not on this page · raw message

Signature · 1 line
2.5.5
#38 lore raw

RFC 6/7 zuf: Filesystem operations

The principle for all operations is the same.

* The few parameters are given on the despatch IOCTL buffer (up to 4k of parameters)

* Any application buffers or other big buffers like readdir are mapped via zuf-core to the the Server VM

* The operation is despatched. Return code and few out parameter are returned in the despatch-return buffer. Any data stored/read at mapped application buffers.

* zus objects like zus_inode symlinks and so on are returned through a dpp_t (Dual port pointer) - a special kind of zuf construct that enables to have a Kernel pointer and a server pointer to the same memory. If pmem is used this is usually a pointer to pmem.

The Kernel's zuf part may even write to this returned pointer but it will then send a despatch, for the FS to persist the change.

TODO:
This patch is probably too big. how best to split it?

---
 fs/zuf/Makefile    |   2 +-
 fs/zuf/_extern.h   |  48 +++++
 fs/zuf/directory.c | 156 ++++++++++++++
 fs/zuf/file.c      | 403 ++++++++++++++++++++++++++++++++++
 fs/zuf/inode.c     | 617 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 fs/zuf/namei.c     | 421 ++++++++++++++++++++++++++++++++++++
 fs/zuf/symlink.c   |  76 +++++++
 fs/zuf/zus_api.h   | 234 ++++++++++++++++++++
 8 files changed, 1953 insertions(+), 4 deletions(-)
 create mode 100644 fs/zuf/directory.c
 create mode 100644 fs/zuf/file.c
 create mode 100644 fs/zuf/namei.c
 create mode 100644 fs/zuf/symlink.c
fs/zuf/Makefile +1 -1
index 94ce80b..4c125f7 100644
@@ -17,5 +17,5 @@ zuf-y += md.o t2.o t1.o
 zuf-y += zuf-core.o zuf-root.o
 
 # Main FS
-zuf-y += super.o inode.o
+zuf-y += super.o inode.o directory.o file.o namei.o symlink.o
 zuf-y += module.o
fs/zuf/_extern.h +48 -0

68 lines not on this page · raw message

fs/zuf/directory.c new file +156 -0

156 lines not on this page · raw message

fs/zuf/file.c new file +403 -0

403 lines not on this page · raw message

fs/zuf/inode.c +614 -3

638 lines not on this page · raw message

fs/zuf/namei.c new file +421 -0

421 lines not on this page · raw message

fs/zuf/symlink.c new file +76 -0

76 lines not on this page · raw message

fs/zuf/zus_api.h +234 -0

250 lines not on this page · raw message

Signature · 1 line
2.5.5
#39 lore raw

RFC 7/7 zuf: Write/Read && mmap implementation

In reply to Boaz Harrosh

The despatch to the server can operate on buffers up to 4 Mega bytes. Any bigger operations are split up and despatched at this size.

Also if a multy-segments aio is used each segment is despatched on its own. (TODO this can be easily fixed with sg operations)

On write if any mmaped buffers changed, for example new allocated holes do to this write or a previous mmaped COW was written. A range subset of the written range can be returned for the Kernel to call mapping_unmap on.

mmap is achived with the GET_BLOCK operation. GET_BLOCK will return if we need to unmap the previous mapped PTE in case we are writing a prevouse faulted hole or in case of a COW.

---
 fs/zuf/Makefile  |   1 +
 fs/zuf/_extern.h |   7 ++
 fs/zuf/file.c    |   3 +
 fs/zuf/mmap.c    | 335 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/zuf/rw.c      | 167 +++++++++++++++++++++++++++
 fs/zuf/zus_api.h |  26 +++++
 6 files changed, 539 insertions(+)
 create mode 100644 fs/zuf/mmap.c
 create mode 100644 fs/zuf/rw.c
fs/zuf/Makefile +1 -0
index 4c125f7..0eb933c 100644
@@ -17,5 +17,6 @@ zuf-y += md.o t2.o t1.o
 zuf-y += zuf-core.o zuf-root.o
 
 # Main FS
+zuf-y += rw.o mmap.o
 zuf-y += super.o inode.o directory.o file.o namei.o symlink.o
 zuf-y += module.o
fs/zuf/_extern.h +7 -0
index cf2e80f..16e99e9 100644
@@ -46,6 +46,13 @@ bool zuf_dir_emit(struct super_block *sb, struct dir_context *ctx,
 uint zuf_prepare_symname(struct zufs_ioc_new_inode *ioc_new_inode,
 			const char *symname, ulong len, struct page *pages[2]);
 
+/* mmap.c */
+int zuf_file_mmap(struct file *file, struct vm_area_struct *vma);
+
+/* rw.c */
+ssize_t zuf_rw_read_iter(struct kiocb *kiocb, struct iov_iter *ii);
+ssize_t zuf_rw_write_iter(struct kiocb *kiocb, struct iov_iter *ii);
+
 /* file.c */
 int zuf_isync(struct inode *inode, loff_t start, loff_t end, int datasync);
 
fs/zuf/file.c +3 -0
index 3b37d9f..3fe59d1 100644
@@ -386,6 +386,9 @@ static int zuf_file_release(struct inode *inode, struct file *filp)
 
 const struct file_operations zuf_file_operations = {
 	.llseek			= zuf_llseek,
+	.read_iter		= zuf_rw_read_iter,
+	.write_iter		= zuf_rw_write_iter,
+	.mmap			= zuf_file_mmap,
 	.open			= generic_file_open,
 	.fsync			= zuf_fsync,
 	.flush			= zuf_flush,
fs/zuf/mmap.c new file +335 -0

335 lines not on this page · raw message

fs/zuf/rw.c new file +167 -0

167 lines not on this page · raw message

fs/zuf/zus_api.h +26 -0
index 5870d63..90b34e4 100644
@@ -529,6 +529,32 @@ static inline bool zufs_zde_emit(struct zufs_readdir_iter *rdi, __u64 ino,
 	return true;
 }
 
+/* ZUS_OP_READ/ZUS_OP_WRITE */
+struct zufs_ioc_IO {
+	struct zufs_ioc_hdr hdr;
+	struct zus_inode_info *zus_ii; /* IN */
+
+	__u64 filepos;
+};
+
+enum {
+	ZUFS_GBF_RESERVED = 1,
+	ZUFS_GBF_NEW = 2,
+};
+
+/* ZUS_OP_GET_BLOCK */
+struct zufs_ioc_get_block {
+	struct zufs_ioc_hdr hdr;
+	 /* IN */
+	struct zus_inode_info *zus_ii;
+	__u64 index; /* page index in file */
+	__u64 rw; /* Some flags + READ or WRITE */
+
+	/* OUT */
+	zu_dpp_t pmem_bn; /* zero return means: map a hole */
+	__u64 ret_flags;  /* One of ZUFS_GBF_XXX */
+};
+
 /* ZUS_OP_GET_SYMLINK */
 struct zufs_ioc_get_link {
 	struct zufs_ioc_hdr hdr;
Signature · 1 line
2.5.5