Skip to contentSkip to conversation

relayfs redux, part 4

PATCH In review Tom Zanussi 5 participants 7 messages Feb 10, 2005 VFS

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 relayfs redux, part 4

Hi,

Here's the latest relayfs patch, incorporating the previous round of suggestions. Thanks to everyone who sent comments. Here's a list of the major changes:

- replaced remap_page_range() and reserved bits with a nopage() handler.
- buffers are now allocated/freed as part of inode alloc/destroy.
- got rid of the automatic creation/teardown of directory hierarchies, and exported create/remove dir functions instead.
- replaced the fileop callback with explicit map/unmap callbacks - these were the only fileops currently being used by anything, so I got rid of the other cases.
- relay_write() and __relay_write() no longer return a value - it doesn't really make sense for clients to check a return value in the fast path anyway.
- added a buf_full() callback to notify users that a buffer has become full and therefore events might be lost.
- got rid of the 'helper' macros, which weren't helping much.
- various other bits of code and comment cleanup.

Also, there was some question as to whether or not the memcpy in relay_write() was being inlined properly - I looked at the generated assembly code, and it seems to be, but I'll be taking a closer look later.

This is what the API now looks like:

API functions:

    rchan *relay_open(base_filename, parent, subbuf_size, n_subbufs,
                      flags, callbacks);
    void relay_close(chan);
    dentry *relayfs_create_dir(name, parent);
    int relayfs_remove_dir(dentry);
    void relay_reset(chan);
  
    void relay_write(chan, data, length);
    void __relay_write(chan, data, length);
    void *relay_reserve(chan, length);

    void relay_subbufs_consumed(chan, subbufs_consumed, cpu);
    void relay_commit(buf, subbuf_idx, count);

callbacks:

    int subbuf_start(buf, subbuf, prev_subbuf_idx);
    int deliver(buffer, subbuf, subbuf_idx);
    void buf_mapped(buf, filp);
    void buf_unmapped(buf, filp);
    void buf_full(buf);

As before, I've tested this code on a single proc machine using a hacked version of the kprobes network packet tracing module, which can be found here:

http://prdownloads.sourceforge.net/dprobes/plog.tar.gz?download

If people are more or less happy with the current version, I'll do some SMP testing and write some Documentation.

Thanks,

Tom

 fs/Kconfig                 |   12 
 fs/Makefile                |    1 
 fs/relayfs/Makefile        |    4 
 fs/relayfs/buffers.c       |  119 ++++++++
 fs/relayfs/buffers.h       |   14 +
 fs/relayfs/inode.c         |  447 ++++++++++++++++++++++++++++++++
 fs/relayfs/relay.c         |  616 +++++++++++++++++++++++++++++++++++++++++++++
 fs/relayfs/relay.h         |   31 ++
 include/linux/relayfs_fs.h |  253 ++++++++++++++++++
 9 files changed, 1497 insertions(+)
fs/Kconfig +12 -0
diff -urpN -X dontdiff linux-2.6.10/fs/Kconfig linux-2.6.10-cur/fs/Kconfig
@@ -972,6 +972,18 @@ config RAMFS
 	  To compile this as a module, choose M here: the module will be called
 	  ramfs.
 
+config RELAYFS_FS
+	tristate "Relayfs file system support"
+	---help---
+	  Relayfs is a high-speed data relay filesystem designed to provide
+	  an efficient mechanism for tools and facilities to relay large
+	  amounts of data from kernel space to user space.
+
+	  To compile this code as a module, choose M here: the module will be
+	  called relayfs.
+
+	  If unsure, say N.
+
 endmenu
 
 menu "Miscellaneous filesystems"
fs/Makefile +1 -0
diff -urpN -X dontdiff linux-2.6.10/fs/Makefile linux-2.6.10-cur/fs/Makefile
@@ -94,3 +94,4 @@ obj-$(CONFIG_AFS_FS)		+= afs/
 obj-$(CONFIG_BEFS_FS)		+= befs/
 obj-$(CONFIG_HOSTFS)		+= hostfs/
 obj-$(CONFIG_HPPFS)		+= hppfs/
+obj-$(CONFIG_RELAYFS_FS)	+= relayfs/
fs/relayfs/Makefile +4 -0
diff -urpN -X dontdiff linux-2.6.10/fs/relayfs/Makefile linux-2.6.10-cur/fs/relayfs/Makefile
@@ -0,0 +1,4 @@
+obj-$(CONFIG_RELAYFS_FS) += relayfs.o
+
+relayfs-y := relay.o buffers.o inode.o
+
fs/relayfs/buffers.c +119 -0

119 lines not on this page · raw message

fs/relayfs/buffers.h +14 -0
diff -urpN -X dontdiff linux-2.6.10/fs/relayfs/buffers.h linux-2.6.10-cur/fs/relayfs/buffers.h
@@ -0,0 +1,14 @@
+#ifndef _BUFFERS_H
+#define _BUFFERS_H
+
+/* This inspired by rtai/shmem */
+#define FIX_SIZE(x) (((x) - 1) & PAGE_MASK) + PAGE_SIZE
+
+extern void *relay_alloc_rchan_buf(unsigned long size,
+				   struct page ***page_array,
+				   int *page_count);
+extern void relay_free_rchan_buf(void *buf,
+				 struct page **page_array,
+				 int page_count);
+
+#endif/* _BUFFERS_H */
fs/relayfs/inode.c +447 -0

447 lines not on this page · raw message

fs/relayfs/relay.c +616 -0

616 lines not on this page · raw message

fs/relayfs/relay.h +31 -0
diff -urpN -X dontdiff linux-2.6.10/fs/relayfs/relay.h linux-2.6.10-cur/fs/relayfs/relay.h
@@ -0,0 +1,31 @@
+/*
+ * fs/relayfs/relay.h
+ *
+ * Copyright (C) 1999-2005 - Karim Yaghmour ([email protected]), Opersys inc.
+ * Copyright (C) 2002, 2003 - Tom Zanussi ([email protected]), IBM Corp
+ *
+ * RelayFS internal definitions and declarations
+ */
+
+#ifndef _RELAY_H
+#define _RELAY_H
+
+/*
+ * Internal relayfs kernel API, fs/relayfs/relay.c
+ */
+
+#define relay_mode_continuous(chan) (((chan)->flags & RELAY_MODE_CONTINUOUS) ? 1 : 0)
+
+struct dentry *relayfs_create_file(const char *name,
+				   struct dentry *parent,
+				   int mode,
+				   struct rchan *chan);
+extern int relayfs_remove(struct dentry *dentry);
+extern int relay_mmap_buffer(struct rchan_buf *buf,
+			     struct vm_area_struct *vma);
+extern struct rchan_buf *rchan_create_buf(struct rchan *chan);
+extern void rchan_destroy_buf(struct rchan_buf *buf);
+extern void relay_destroy_buf(struct kref *kref);
+extern int relay_buf_empty(struct rchan_buf *buf);
+
+#endif /* _RELAY_H */
include/linux/relayfs_fs.h +253 -0

253 lines not on this page · raw message

#1 lore raw

On Wed, Feb 09, 2005 at 08:49:36PM -0600, Tom Zanussi wrote:
[..]

fs/relayfs/inode.c line 179

Quoted 12 lines

+ */

fs/relayfs/inode.c +11 -0
+struct dentry *relayfs_create_file(const char *name, struct dentry *parent,
+				   int mode, struct rchan *chan)
+{
+	struct dentry *dentry;
+	int error;
+	
+	if (!mode)
+		mode = S_IRUSR;
+	mode = (mode & S_IALLUGO) | S_IFREG;
+
+	error = relayfs_create_entry(name, parent, mode, chan, &dentry);

^^^^
I think you missed GregKH's suggesstion to have relayfs_create_entry() return pointer to struct dentry, and reduce one parameter.

Quoted 12 lines
fs/relayfs/relay.c +12 -0
+unsigned relay_switch_subbuf(struct rchan_buf *buf, unsigned length)
+{
+	int cur_subbuf, prev_subbuf, subbufs_produced;
+	unsigned start = 0, padding = buf->chan->subbuf_size - buf->offset;
+	
+	if (unlikely(relay_buf_full(buf)))
+		return 0;
+
+	subbufs_produced = atomic_read(&buf->subbufs_produced);
+	cur_subbuf = prev_subbuf = subbufs_produced % buf->chan->n_subbufs;
+	buf->padding[cur_subbuf] = padding;
+

fs/relayfs/relay.c line 518

Quoted 10 lines
fs/relayfs/relay.c +10 -0
+	atomic_inc(&buf->subbufs_produced);
+
+	if (waitqueue_active(&buf->read_wait)) {
+		PREPARE_WORK(&buf->wake_readers, wakeup_readers, buf);
+		schedule_delayed_work(&buf->wake_readers, 1);
+	}
+
+	if (unlikely(relay_buf_full(buf))) {
+		return 0;
+		buf->chan->cb->buf_full(buf);
		^^^^^^^^
		Typo? statement after return !

Thanks
Maneesh

Signature · 6 lines
Maneesh Soni
Linux Technology Center, 
IBM India Software Labs,
Bangalore, India
email: [email protected]
Phone: 91-80-25044990
#2 lore raw
Quoted 18 lines · Maneesh Soni

On Wed, Feb 09, 2005 at 08:49:36PM -0600, Tom Zanussi wrote:
[..]

fs/relayfs/inode.c line 179

Quoted 12 lines

+ */

fs/relayfs/inode.c +11 -0
+struct dentry *relayfs_create_file(const char *name, struct dentry *parent,
+				   int mode, struct rchan *chan)
+{
+	struct dentry *dentry;
+	int error;
+	
+	if (!mode)
+		mode = S_IRUSR;
+	mode = (mode & S_IALLUGO) | S_IFREG;
+
+	error = relayfs_create_entry(name, parent, mode, chan, &dentry);

^^^^
I think you missed GregKH's suggesstion to have relayfs_create_entry() return pointer to struct dentry, and reduce one parameter.

Yes, you're right - somehow I missed that one.

Quoted 7 lines

fs/relayfs/relay.c line 518

Quoted 4 lines
fs/relayfs/relay.c +4 -0
+
+	if (unlikely(relay_buf_full(buf))) {
+		return 0;
+		buf->chan->cb->buf_full(buf);
		^^^^^^^^
		Typo? statement after return !

Yikes! Obviously I haven't tested the buffer full condition yet ;-)

Thanks for pointing these out.

Tom

#3 lore raw
In reply to Tom Zanussi

Hi Tom,

Quoted 15 lines · Tom Zanussi
fs/relayfs/buffers.c +15 -0
+static inline struct page **alloc_page_array(int size, int *page_count)
+{
+       int n_pages;
+       struct page **page_array;
+
+       size = PAGE_ALIGN(size);
+       n_pages = size >> PAGE_SHIFT;
+
+       page_array = kcalloc(n_pages, sizeof(struct page *), GFP_KERNEL);
+       if (!page_array)
+               return NULL;
+       *page_count = n_pages;
+
+       return page_array;
+}

[snip]

Quoted 26 lines
fs/relayfs/buffers.c +26 -0
+static int populate_page_array(struct page **page_array, int page_count)
+{
+       int i;
+
+       for (i = 0; i < page_count; i++) {
+               page_array[i] = alloc_page(GFP_KERNEL);
+               if (unlikely(!page_array[i])) {
+                       depopulate_page_array(page_array, i);
+                       return -ENOMEM;
+               }
+       }
+       return 0;
+}
+
+/**
+ *     relay_alloc_rchan_buf - allocate a channel buffer
+ *     @size: total size of the buffer
+ *     @page_array: receives a pointer to the buffer's page array
+ *     @page_count: receives the number of pages allocated
+ *
+ *     Returns a pointer to the resulting buffer, NULL if unsuccessful
+ */
+void *relay_alloc_rchan_buf(unsigned long size, struct page ***page_array,
+                           int *page_count)
+{
+       void *mem;

fs/relayfs/buffers.c line 93

Quoted 10 lines
fs/relayfs/buffers.c +10 -0
+
+       *page_array = alloc_page_array(size, page_count);
+       if (!*page_array)
+               return NULL;
+
+       if (populate_page_array(*page_array, *page_count)) {
+               kfree(*page_array);
+               *page_array = NULL;
+               return NULL;
+       }

[snip]

Please consider inlining alloc_page_array() and populate_page_array() into relay_alloc_rchan_buf() as they're only used once. You'd get rid of passing page_count as a pointer this way. If inlining is unacceptable, please at least move the n_pages calculation to relay_alloc_rchan_buf() to make the API more sane.

I think relay_alloc_rchan_buf also would benefit from goto-styled error handling.

Pekka

#4 lore raw
Quoted 6 lines

Please consider inlining alloc_page_array() and populate_page_array() into relay_alloc_rchan_buf() as they're only used once. You'd get rid of passing page_count as a pointer this way. If inlining is unacceptable, please at least move the n_pages calculation to relay_alloc_rchan_buf() to make the API more sane.

Modern gcc (3.3-hammer with unit-at-a-time or 3.4) will do that anyways on its own.

-Andi

#5 lore raw
Quoted 5 lines · I wrote:
Quoted 5 lines

Please consider inlining alloc_page_array() and populate_page_array() into relay_alloc_rchan_buf() as they're only used once. You'd get rid of passing page_count as a pointer this way. If inlining is unacceptable, please at least move the n_pages calculation to relay_alloc_rchan_buf() to make the API more sane.

Quoted 1 line · Andi Kleen

Modern gcc (3.3-hammer with unit-at-a-time or 3.4) will do that anyways on its own.

I know that but I am commenting on readability. The pointer passing is due to silly interface (alloc_page_array() calculating number of pages on its own).

Pekka

#6 lore raw
In reply to Tom Zanussi
Quoted 14 lines · Tom Zanussi
fs/relayfs/inode.c +14 -0
+static int relayfs_create_entry(const char *name, struct dentry *parent,
+                               int mode, struct rchan *chan,
+                               struct dentry **dentry)
+{
+       struct qstr qname;
+       struct dentry *d;
+       struct inode *inode;
+       int error = 0;
+
+       BUG_ON(!(S_ISREG(mode) || S_ISDIR(mode)));
+
+       error = simple_pin_fs("relayfs", &relayfs_mount, &relayfs_mount_count);
+       if (error) {
+               printk(KERN_ERR "Couldn't mount relayfs: errcode %d\n", error);

fs/relayfs/inode.c line 113

Quoted 10 lines
fs/relayfs/inode.c +10 -0
+               return error;
+       }
+
+       qname.name = name;
+       qname.len = strlen(name);
+       qname.hash = full_name_hash(name, qname.len);
+
+       if (!parent)
+               if (relayfs_mount && relayfs_mount->mnt_sb)
+                       parent = relayfs_mount->mnt_sb->s_root;

Please move the nested if statement to the parent expression. The !parent part is always evaluated first.

fs/relayfs/inode.c line 331

Quoted 5 lines
fs/relayfs/inode.c +5 -0
+static struct inode *relayfs_alloc_inode(struct super_block *sb)
+{
+       struct relayfs_inode_info *p;
+       p = (struct relayfs_inode_info *)kmem_cache_alloc(relayfs_inode_cachep,
+                                                         SLAB_KERNEL);

Please drop the spurious cast from void *.

Pekka