Skip to contentSkip to conversation

fs: Add VirtualBox guest shared folder (vboxsf) support

PATCH Superseded Hans de Goede 2 participants 4 messages Mar 7, 2019 – Mar 8, 2019

Subsystems (2)

v8 of 19 Superseded Latest is v19 →

lore
Share & tools
lore
#0 lore raw

v8 cover fs: Add VirtualBox guest shared folder (vboxsf) support

Hi All,

Here is the 8th version of my cleaned-up / refactored version of the VirtualBox shared-folder VFS driver.

This version has been used by several distributions (arch, Fedora) for a while now and as such has seen quite a bit of testing. This testing has found 2 issues which are both fixed in this new version.

Since the whole purpose of the clean-up / refactoring was to make this code suitable for mainline inclusion and since this is being quite widely used now, it would be nice if this version could get an upstream review and then after 1 or 2 more revisions it can hopefully be merged.

The changes between versions are detailed in the commit msg of the patch.

Regards,

Hans

#1 lore raw

PATCH v8 1/1 fs: Add VirtualBox guest shared folder (vboxsf) support

VirtualBox hosts can share folders with guests, this commit adds a VFS driver implementing the Linux-guest side of this, allowing folders exported by the host to be mounted under Linux.

This driver depends on the guest <-> host IPC functions exported by the vboxguest driver.

---
Changes in v8:
-Fix broken error-handling / oops when the vboxsf_map_folder() call fails -Fix umount using umount.nfs to umount vboxsf mounts
-Prefixed the modules init and exit function names with vboxsf_ -Delay connecting to the vbox hypervisor until the first mount, this fixes
vboxsf not working when it is builtin (in which case it may be initialized before the vboxguest driver has bound to the guest communication PCI device)
-Fix sf_write_end return value, return the number of bytes written or 0 on error:
https://github.com/jwrdegoede/vboxsf/issues/2
-Use an ida id in the name passed to super_setup_bdi_name so that the same
shared-folder can be mounted twice without causing a
"sysfs: cannot create duplicate filename" error
https://github.com/jwrdegoede/vboxsf/issues/3

Changes in v7:
-Do not propagate sgid / suid bits between guest-host, note hosts with
VirtualBox version 5.2.6 or newer will filter these out regardless of what we do
-Better error messages when we cannot connect to the VirtualBox guest PCI
device, which may e.g. happen when trying to use vboxsf outside a vbox vm

Changes in v6:
-Address: https://www.virtualbox.org/ticket/819 which really is multiple bugs:
1) Fix MAP_SHARED not being supported
2) Fix changes done through regular read/write on the guest side not being seen by guest apps using mmap() access
3) Fix any changes done on the host side not being seen by guest apps using mmap() access

Changes in v5:
-Honor CONFIG_NLS_DEFAULT (reported-by [email protected])

Changes in v4:
-Drop "name=..." mount option, instead use the dev_name argument to the
mount syscall, to keep compatibility with existing fstab entries
-Fix "nls=%" match_table_t entry to "nls=%s"

Changes in v3:
-Use text only mount options, instead of a custom data struct -Stop caching full path in inode data, if parents gets renamed it will change -Fixed negative dentries handling
-Dropped the force_reread flag for dirs, not sure what it was actually for
but it is no good, doing a re-read on unlink of a file will lead to another file being skipped if the caller has already iterated over the entry for the unlinked file.
-Use file_inode(), file_dentry() and d_inode() helpers
-Prefix any non object-private symbols with vboxsf_ so as to not pollute
the global namespace when builtin
-Add MAINTAINERS entry
-Misc. cleanups

Changes in v2:
-Removed various unused wrapper functions
-Don't use i_private, instead defined alloc_inode and destroy_inode
methods and use container_of.
-Drop obsolete comment referencing people to
http://www.atnf.csiro.au/people/rgooch/linux/vfs.txt
-move the single symlink op of from lnkops.c to file.c
-Use SPDX license headers
-Replace SHFLROOT / SHFLHANDLE defines with normal types -Removed unnecessary S_ISREG checks
-Got rid of bounce_buffer in regops, instead add a "user" flag to
vboxsf_read / vboxsf_write, re-using the existing __user address support in the vboxguest module
-Make vboxsf_wrappers return regular linux errno values
-Use i_size_write to update size on writing
-Convert doxygen style comments to kerneldoc style comments

---
 MAINTAINERS                 |   6 +
 fs/Kconfig                  |   1 +
 fs/Makefile                 |   1 +
 fs/vboxsf/Kconfig           |  10 +
 fs/vboxsf/Makefile          |   3 +
 fs/vboxsf/dir.c             | 574 ++++++++++++++++++++++
 fs/vboxsf/file.c            | 369 +++++++++++++++
 fs/vboxsf/shfl_hostintf.h   | 919 ++++++++++++++++++++++++++++++++++++
 fs/vboxsf/super.c           | 478 +++++++++++++++++++
 fs/vboxsf/utils.c           | 520 ++++++++++++++++++++
 fs/vboxsf/vboxsf_wrappers.c | 370 +++++++++++++++
 fs/vboxsf/vfsmod.h          | 134 ++++++
 12 files changed, 3385 insertions(+)
 create mode 100644 fs/vboxsf/Kconfig
 create mode 100644 fs/vboxsf/Makefile
 create mode 100644 fs/vboxsf/dir.c
 create mode 100644 fs/vboxsf/file.c
 create mode 100644 fs/vboxsf/shfl_hostintf.h
 create mode 100644 fs/vboxsf/super.c
 create mode 100644 fs/vboxsf/utils.c
 create mode 100644 fs/vboxsf/vboxsf_wrappers.c
 create mode 100644 fs/vboxsf/vfsmod.h
MAINTAINERS +6 -0
index dce5c099f43c..81d484891b53 100644
@@ -16316,6 +16316,12 @@ F:	include/linux/vbox_utils.h
 F:	include/uapi/linux/vbox*.h
 F:	drivers/virt/vboxguest/
 
+VIRTUAL BOX SHARED FOLDER VFS DRIVER:
+M:	Hans de Goede <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	fs/vboxsf/*
+
 VIRTUAL SERIO DEVICE DRIVER
 M:	Stephen Chandler Paul <[email protected]>
 S:	Maintained
fs/Kconfig +1 -0
index ac474a61be37..b1b7e23b96ec 100644
@@ -255,6 +255,7 @@ source "fs/pstore/Kconfig"
 source "fs/sysv/Kconfig"
 source "fs/ufs/Kconfig"
 source "fs/exofs/Kconfig"
+source "fs/vboxsf/Kconfig"
 
 endif # MISC_FILESYSTEMS
 
fs/Makefile +1 -0
index 293733f61594..edc288bd8654 100644
@@ -128,3 +128,4 @@ obj-y				+= exofs/ # Multiple modules
 obj-$(CONFIG_CEPH_FS)		+= ceph/
 obj-$(CONFIG_PSTORE)		+= pstore/
 obj-$(CONFIG_EFIVAR_FS)		+= efivarfs/
+obj-$(CONFIG_VBOXSF_FS)		+= vboxsf/
fs/vboxsf/Kconfig new file +10 -0
new file mode 100644
index 000000000000..b034698121ea
@@ -0,0 +1,10 @@
+config VBOXSF_FS
+	tristate "VirtualBox guest shared folder (vboxsf) support"
+	depends on VBOXGUEST
+	select NLS
+	help
+	  VirtualBox hosts can share folders with guests, this driver
+	  implements the Linux-guest side of this allowing folders exported
+	  by the host to be mounted under Linux.
+
+	  If you want to use shared folders in VirtualBox guests, answer Y or M.
fs/vboxsf/Makefile new file +3 -0
new file mode 100644
index 000000000000..fcbd488f9eec
@@ -0,0 +1,3 @@
+obj-$(CONFIG_VBOXSF_FS) += vboxsf.o
+
+vboxsf-objs := dir.o file.o utils.o vboxsf_wrappers.o super.o
fs/vboxsf/dir.c new file +574 -0

574 lines not on this page · raw message

fs/vboxsf/file.c new file +369 -0

369 lines not on this page · raw message

fs/vboxsf/shfl_hostintf.h new file +919 -0

919 lines not on this page · raw message

fs/vboxsf/super.c new file +478 -0

478 lines not on this page · raw message

fs/vboxsf/utils.c new file +520 -0

520 lines not on this page · raw message

fs/vboxsf/vboxsf_wrappers.c new file +370 -0

104 lines not on this page · raw message

fs/vboxsf/vfsmod.h new file +134 -0

134 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.20.1

Part of the diff is not shown here. View the raw message.

#2 lore raw
I love your patch! Perhaps something to improve:

Hi Hans,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on v5.0]
[cannot apply to next-20190306]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url: https://github.com/0day-ci/linux/commits/Hans-de-Goede/fs-Add-VirtualBox-guest-shared-folder-vboxsf-support/20190307-231837
reproduce:

        # apt-get install sparse
        make ARCH=x86_64 allmodconfig
        make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'

All warnings (new ones prefixed by >>):

>> fs/vboxsf/file.c:215:17: sparse: warning: incorrect type in initializer (incompatible argument 2 (different address spaces))
   fs/vboxsf/file.c:215:17: sparse:    expected long ( *read )( ... )
   fs/vboxsf/file.c:215:17: sparse:    got long ( * )( ... )
   fs/vboxsf/file.c:217:18: sparse: warning: incorrect type in initializer (incompatible argument 2 (different address spaces))
   fs/vboxsf/file.c:217:18: sparse:    expected long ( *write )( ... )
   fs/vboxsf/file.c:217:18: sparse:    got long ( * )( ... )
>> fs/vboxsf/file.c:288:5: sparse: warning: symbol 'sf_write_end' was not declared. Should it be static?

sparse warnings: (new ones prefixed by >>)

fs/vboxsf/file.c:215:17: sparse: warning: incorrect type in initializer (incompatible argument 2 (different address spaces))

>> fs/vboxsf/file.c:215:17: sparse:    expected long ( *read )( ... )
>> fs/vboxsf/file.c:215:17: sparse:    got long ( * )( ... )
   fs/vboxsf/file.c:217:18: sparse: warning: incorrect type in initializer (incompatible argument 2 (different address spaces))
>> fs/vboxsf/file.c:217:18: sparse:    expected long ( *write )( ... )

fs/vboxsf/file.c:217:18: sparse: got long ( * )( ... )
fs/vboxsf/file.c:288:5: sparse: warning: symbol 'sf_write_end' was not declared. Should it be static?

vim +215 fs/vboxsf/file.c

213
214 const struct file_operations vboxsf_reg_fops = {

 > 215		.read = sf_reg_read,
   216		.open = sf_reg_open,
 > 217		.write = sf_reg_write,
   218		.release = sf_reg_release,
   219		.mmap = generic_file_mmap,
   220		.splice_read = generic_file_splice_read,
   221		.read_iter = generic_file_read_iter,
   222		.write_iter = generic_file_write_iter,
   223		.fsync = noop_fsync,
   224		.llseek = generic_file_llseek,
   225	};
   226	
   227	const struct inode_operations vboxsf_reg_iops = {
   228		.getattr = vboxsf_getattr,
   229		.setattr = vboxsf_setattr
   230	};
   231	
   232	static int sf_readpage(struct file *file, struct page *page)
   233	{
   234		struct sf_glob_info *sf_g = GET_GLOB_INFO(file_inode(file)->i_sb);
   235		struct sf_reg_info *sf_r = file->private_data;
   236		loff_t off = page_offset(page);
   237		u32 nread = PAGE_SIZE;
   238		u8 *buf;
   239		int err;
   240	
   241		buf = kmap(page);
   242	
   243		err = vboxsf_read(sf_g->root, sf_r->handle, off, &nread, buf, false);
   244		if (err == 0) {
   245			memset(&buf[nread], 0, PAGE_SIZE - nread);
   246			flush_dcache_page(page);
   247			SetPageUptodate(page);
   248		} else {
   249			SetPageError(page);
   250		}
   251	
   252		kunmap(page);
   253		unlock_page(page);
   254		return err;
   255	}
   256	
   257	static int sf_writepage(struct page *page, struct writeback_control *wbc)
   258	{
   259		struct inode *inode = page->mapping->host;
   260		struct sf_glob_info *sf_g = GET_GLOB_INFO(inode->i_sb);
   261		struct sf_inode_info *sf_i = GET_INODE_INFO(inode);
   262		struct sf_reg_info *sf_r = sf_i->file->private_data;
   263		loff_t off = page_offset(page);
   264		loff_t size = i_size_read(inode);
   265		u32 nwrite = PAGE_SIZE;
   266		u8 *buf;
   267		int err;
   268	
   269		if (off + PAGE_SIZE > size)
   270			nwrite = size & ~PAGE_MASK;
   271	
   272		buf = kmap(page);
   273		err = vboxsf_write(sf_g->root, sf_r->handle, off, &nwrite, buf, false);
   274		kunmap(page);
   275	
   276		if (err == 0) {
   277			ClearPageError(page);
   278			/* mtime changed */
   279			sf_i->force_restat = 1;
   280		} else {
   281			ClearPageUptodate(page);
   282		}
   283	
   284		unlock_page(page);
   285		return err;
   286	}
   287	
 > 288	int sf_write_end(struct file *file, struct address_space *mapping, loff_t pos,

289 unsigned int len, unsigned int copied, struct page *page,
290 void *fsdata)
291 {
292 struct inode *inode = mapping->host;
293 struct sf_glob_info *sf_g = GET_GLOB_INFO(inode->i_sb);
294 struct sf_reg_info *sf_r = file->private_data;
295 unsigned int from = pos & ~PAGE_MASK;
296 u32 nwritten = len;
297 u8 *buf;
298 int err;
299
300 buf = kmap(page);
301 err = vboxsf_write(sf_g->root, sf_r->handle, pos, &nwritten,
302 buf + from, false);
303 kunmap(page);
304
305 if (err) {
306 nwritten = 0;
307 goto out;
308 }
309
310 /* mtime changed */
311 GET_INODE_INFO(inode)->force_restat = 1;
312
313 if (!PageUptodate(page) && nwritten == PAGE_SIZE)
314 SetPageUptodate(page);
315
316 pos += nwritten;
317 if (pos > inode->i_size)
318 i_size_write(inode, pos);
319
320 out:
321 unlock_page(page);
322 put_page(page);
323
324 return nwritten;
325 }
326

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

#3 lore raw
In reply to Hans de Goede
I love your patch! Yet something to improve:

Hi Hans,

I love your patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.0]
[cannot apply to next-20190306]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url: https://github.com/0day-ci/linux/commits/Hans-de-Goede/fs-Add-VirtualBox-guest-shared-folder-vboxsf-support/20190307-231837
config: x86_64-randconfig-s4-03081928 (attached as .config)
compiler: gcc-8 (Debian 8.3.0-2) 8.3.0
reproduce:

        # save the attached .config to linux build tree
        make ARCH=x86_64 

All error/warnings (new ones prefixed by >>):

fs/vboxsf/file.c: In function 'sf_readpage':

>> fs/vboxsf/file.c:236:15: error: implicit declaration of function 'page_offset'; did you mean 'gate_offset'? [-Werror=implicit-function-declaration]
     loff_t off = page_offset(page);
                  ^~~~~~~~~~~
                  gate_offset
>> fs/vboxsf/file.c:241:8: error: implicit declaration of function 'kmap'; did you mean 'swap'? [-Werror=implicit-function-declaration]
     buf = kmap(page);
           ^~~~
           swap
>> fs/vboxsf/file.c:241:6: warning: assignment to 'u8 *' {aka 'unsigned char *'} from 'int' makes pointer from integer without a cast [-Wint-conversion]
     buf = kmap(page);
         ^
>> fs/vboxsf/file.c:246:3: error: implicit declaration of function 'flush_dcache_page'; did you mean 'write_cache_pages'? [-Werror=implicit-function-declaration]
      flush_dcache_page(page);
      ^~~~~~~~~~~~~~~~~
      write_cache_pages
>> fs/vboxsf/file.c:247:3: error: implicit declaration of function 'SetPageUptodate' [-Werror=implicit-function-declaration]
      SetPageUptodate(page);
      ^~~~~~~~~~~~~~~
>> fs/vboxsf/file.c:249:3: error: implicit declaration of function 'SetPageError' [-Werror=implicit-function-declaration]
      SetPageError(page);
      ^~~~~~~~~~~~
>> fs/vboxsf/file.c:252:2: error: implicit declaration of function 'kunmap' [-Werror=implicit-function-declaration]
     kunmap(page);
     ^~~~~~
>> fs/vboxsf/file.c:253:2: error: implicit declaration of function 'unlock_page'; did you mean 'alloc_page'? [-Werror=implicit-function-declaration]
     unlock_page(page);
     ^~~~~~~~~~~
     alloc_page
   fs/vboxsf/file.c: In function 'sf_writepage':
   fs/vboxsf/file.c:272:6: warning: assignment to 'u8 *' {aka 'unsigned char *'} from 'int' makes pointer from integer without a cast [-Wint-conversion]
     buf = kmap(page);
         ^
>> fs/vboxsf/file.c:277:3: error: implicit declaration of function 'ClearPageError'; did you mean 'clear_page_erms'? [-Werror=implicit-function-declaration]
      ClearPageError(page);
      ^~~~~~~~~~~~~~
      clear_page_erms
>> fs/vboxsf/file.c:281:3: error: implicit declaration of function 'ClearPageUptodate' [-Werror=implicit-function-declaration]
      ClearPageUptodate(page);
      ^~~~~~~~~~~~~~~~~
   fs/vboxsf/file.c: In function 'sf_write_end':
   fs/vboxsf/file.c:300:6: warning: assignment to 'u8 *' {aka 'unsigned char *'} from 'int' makes pointer from integer without a cast [-Wint-conversion]
     buf = kmap(page);
         ^
>> fs/vboxsf/file.c:313:7: error: implicit declaration of function 'PageUptodate' [-Werror=implicit-function-declaration]
     if (!PageUptodate(page) && nwritten == PAGE_SIZE)
          ^~~~~~~~~~~~
>> fs/vboxsf/file.c:322:2: error: implicit declaration of function 'put_page'; did you mean 'pgd_page'? [-Werror=implicit-function-declaration]
     put_page(page);
     ^~~~~~~~
     pgd_page
   fs/vboxsf/file.c: At top level:
>> fs/vboxsf/file.c:330:20: error: '__set_page_dirty_nobuffers' undeclared here (not in a function); did you mean 'irq_set_affinity_notifier'?

.set_page_dirty = __set_page_dirty_nobuffers,
^~~~~~~~~~~~~~~~~~~~~~~~~~ irq_set_affinity_notifier
cc1: some warnings being treated as errors

vim +236 fs/vboxsf/file.c

231
232 static int sf_readpage(struct file *file, struct page *page)
233 {
234 struct sf_glob_info *sf_g = GET_GLOB_INFO(file_inode(file)->i_sb);
235 struct sf_reg_info *sf_r = file->private_data;

 > 236		loff_t off = page_offset(page);
   237		u32 nread = PAGE_SIZE;
   238		u8 *buf;
   239		int err;
   240	
 > 241		buf = kmap(page);
   242	
   243		err = vboxsf_read(sf_g->root, sf_r->handle, off, &nread, buf, false);
   244		if (err == 0) {
   245			memset(&buf[nread], 0, PAGE_SIZE - nread);
 > 246			flush_dcache_page(page);
 > 247			SetPageUptodate(page);
   248		} else {
 > 249			SetPageError(page);
   250		}
   251	
 > 252		kunmap(page);
 > 253		unlock_page(page);
   254		return err;
   255	}
   256	
   257	static int sf_writepage(struct page *page, struct writeback_control *wbc)
   258	{
   259		struct inode *inode = page->mapping->host;
   260		struct sf_glob_info *sf_g = GET_GLOB_INFO(inode->i_sb);
   261		struct sf_inode_info *sf_i = GET_INODE_INFO(inode);
   262		struct sf_reg_info *sf_r = sf_i->file->private_data;
   263		loff_t off = page_offset(page);
   264		loff_t size = i_size_read(inode);
   265		u32 nwrite = PAGE_SIZE;
   266		u8 *buf;
   267		int err;
   268	
   269		if (off + PAGE_SIZE > size)
   270			nwrite = size & ~PAGE_MASK;
   271	
 > 272		buf = kmap(page);
   273		err = vboxsf_write(sf_g->root, sf_r->handle, off, &nwrite, buf, false);
   274		kunmap(page);
   275	
   276		if (err == 0) {
 > 277			ClearPageError(page);
   278			/* mtime changed */
   279			sf_i->force_restat = 1;
   280		} else {
 > 281			ClearPageUptodate(page);
   282		}
   283	
   284		unlock_page(page);
   285		return err;
   286	}
   287	
   288	int sf_write_end(struct file *file, struct address_space *mapping, loff_t pos,
   289			 unsigned int len, unsigned int copied, struct page *page,
   290			 void *fsdata)
   291	{
   292		struct inode *inode = mapping->host;
   293		struct sf_glob_info *sf_g = GET_GLOB_INFO(inode->i_sb);
   294		struct sf_reg_info *sf_r = file->private_data;
   295		unsigned int from = pos & ~PAGE_MASK;
   296		u32 nwritten = len;
   297		u8 *buf;
   298		int err;
   299	
 > 300		buf = kmap(page);
   301		err = vboxsf_write(sf_g->root, sf_r->handle, pos, &nwritten,
   302				   buf + from, false);
   303		kunmap(page);
   304	
   305		if (err) {
   306			nwritten = 0;
   307			goto out;
   308		}
   309	
   310		/* mtime changed */
   311		GET_INODE_INFO(inode)->force_restat = 1;
   312	
 > 313		if (!PageUptodate(page) && nwritten == PAGE_SIZE)
   314			SetPageUptodate(page);
   315	
   316		pos += nwritten;
   317		if (pos > inode->i_size)
   318			i_size_write(inode, pos);
   319	
   320	out:
   321		unlock_page(page);
 > 322		put_page(page);
   323	
   324		return nwritten;
   325	}
   326	
   327	const struct address_space_operations vboxsf_reg_aops = {
   328		.readpage = sf_readpage,
   329		.writepage = sf_writepage,
 > 330		.set_page_dirty = __set_page_dirty_nobuffers,

331 .write_begin = simple_write_begin,
332 .write_end = sf_write_end,
333 };
334

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation