Skip to contentSkip to conversation

UnionFS over NFS over UnionFS

DISCUSSION Wilhelm Meier 2 participants 2 messages Dec 9, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hello everybody,

the below problem was allready posted to the unionfs-list, but remains=20 unsolved:
=2D-------------

To clarify things further, this happens only in the follwing scenario (as described in previous posting, see below):

1) we build a union mounted on server:/tftproot/gentoo_B
2) we export this union via NFS
3) we mount server:/tftproot/gentoo_B onto client:/mnt/test/N
4) we build a union on the client consisting of /mnt/test/N an some tmpfs

If we change step 2) to export some local filesytem (tftproot/gentoo_A in the below example) and change step 3) to mount server:/tftproot/gentoo_A onto the client, all works well.

A saw a posting on this list reporting similar problems. Are the any hints?

=2D-
Wilhelm

Quoted 7 lines

Hello,

thank you for the advice, but this does not solve the problem.

On the other hand, I tried to export and nfs-mount all rw, but to declare the nfs-mounted branch ro for the union-mount:

Quoted 16 lines

gs test # mount
/dev/hda1 on / type ext3 (rw,noatime)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
udev on /dev type tmpfs (rw,nosuid)
devpts on /dev/pts type devpts (rw)
/dev/hdb1 on /tftproot type ext3 (rw,noatime)
none on /tftproot/gentoo_B type unionfs (rw,dirs=3D/tftproot/gentoo_Bdiff=3Drw:/tftproot/gentoo_A=3Dro) shm on /dev/shm type tmpfs (rw,noexec,nosuid,nodev)
nfsd on /proc/fs/nfs type nfsd (rw)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) 192.168.39.10:/tftproot/gentoo_B on /mnt/test/N type nfs (rw,addr=3D192.168.39.10)
none on /mnt/test/T type tmpfs (rw)
none on /mnt/test/A type unionfs (rw,dirs=3D/mnt/test/T=3Drw:/mnt/test/N=

=3Dro)

Quoted 4 lines

gs test # touch A/x
touch: setting times of `A/x': Operation not permitted

Quoted 8 lines

gs test # ls A
bin =A0 dev =A0home =A0mnt =A0proc =A0sbin =A0 =A0 =A0 =A0 =A0sys =A0usr boot =A0etc =A0lib =A0 opt =A0root =A0stateless.sh =A0tmp =A0var

gs test # touch N/x

gs test # ls A
bin =A0 dev =A0home =A0mnt =A0proc =A0sbin =A0 =A0 =A0 =A0 =A0sys =A0usr =

=A0x

Quoted 86 lines

boot =A0etc =A0lib =A0 opt =A0root =A0stateless.sh =A0tmp =A0var

gs test # touch A/x
touch: setting times of `A/x': Operation not permitted gs test #

Any hints?

-
Wilhelm

Quoted 73 lines

Hi!!

Quoted 17 lines

The same problem arises also a simpler setup (on the a single machine):

gs ~ # mount
/dev/hda1 on / type ext3 (rw,noatime)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
udev on /dev type tmpfs (rw,nosuid)
devpts on /dev/pts type devpts (rw)
/dev/hdb1 on /tftproot type ext3 (rw,noatime)
shm on /dev/shm type tmpfs (rw,noexec,nosuid,nodev) nfsd on /proc/fs/nfs type nfsd (rw)
/dev/hdc on /mnt/cdrom type iso9660 (ro)
none on /tftproot/gentoo_B type unionfs (rw,dirs=3D/tftproot/gentoo_Bdiff=3Drw:/tftproot/gentoo_A=3Dro) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) 192.168.39.10:/tftproot/gentoo_B on /mnt/test/N type nfs (rw,addr=3D192.168.39.10)

If this share is exported RO it should be mounted RO only. Unfortunately I
may mount a RO nfs share with RW option on a client (of course it does not
renders the share writeable) - but it will confuse unionfs. If you mount your share explicitly RO you shouldn't get a permission denied...

We had the same problem with RO nfs root filesystem unioned with RW tempfs
on top of it. Hope I got the idea right what you intend to do :-))

Ciao,
=A0=A0=A0=A0=A0Dirk

Quoted 37 lines

none on /mnt/test/T type tmpfs (rw)
none on /mnt/test/A type unionfs
(rw,dirs=3D/mnt/test/T=3Drw:/mnt/test/N=3Dro)
gs ~ #

As you see 192.168.39.10:/tftproot/gentoo_B is exported and mounted on /mnt/test/N. This is union-mounted together with tmpfs /mnt/test/T on /mnt/test/A.

On /mnt/test/A we get:

gs A # touch x
touch: setting times of `x': Operation not permitted

Am Montag, 5. Dezember 2005 13:48 schrieben Sie:

Quoted 11 lines
Quoted 4 lines

NFS: Buggy server - nlink =3D=3D 0!
nfs_fhget: iget failed

We are using unionfs-1.0.14

Step 1: Try updating to 1.1.1 or some CVS snapshot.

Jan Engelhardt

--
--
Wilhelm Meier
email: [email protected]
_______________________________________________
unionfs mailing list
[email protected]
http://www.fsl.cs.sunysb.edu/mailman/listinfo/unionfs

=2D-=20
=2D-
Wilhelm Meier
email: [email protected]

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#1 lore raw
Quoted 14 lines · Wilhelm Meier

Hello everybody,

the below problem was allready posted to the unionfs-list, but remains unsolved:
--------------

To clarify things further, this happens only in the follwing scenario (as described in previous posting, see below):

1) we build a union mounted on server:/tftproot/gentoo_B
2) we export this union via NFS
3) we mount server:/tftproot/gentoo_B onto client:/mnt/test/N
4) we build a union on the client consisting of /mnt/test/N an some tmpfs

Quoted 5 lines

If we change step 2) to export some local filesytem (tftproot/gentoo_A in the below example) and change step 3) to mount server:/tftproot/gentoo_A onto the client, all works well.

A saw a posting on this list reporting similar problems. Are the any hints?

You don't actually tell us what the /etc/exports looks like on the server, but given that the test was made from a root account on the client, could it simply be root squashing?

Cheers,
Trond

Quoted 125 lines

--
Wilhelm

Quoted 121 lines

Hello,

thank you for the advice, but this does not solve the problem.

On the other hand, I tried to export and nfs-mount all rw, but to declare the nfs-mounted branch ro for the union-mount:

gs test # mount
/dev/hda1 on / type ext3 (rw,noatime)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
udev on /dev type tmpfs (rw,nosuid)
devpts on /dev/pts type devpts (rw)
/dev/hdb1 on /tftproot type ext3 (rw,noatime)
none on /tftproot/gentoo_B type unionfs (rw,dirs=/tftproot/gentoo_Bdiff=rw:/tftproot/gentoo_A=ro) shm on /dev/shm type tmpfs (rw,noexec,nosuid,nodev)
nfsd on /proc/fs/nfs type nfsd (rw)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) 192.168.39.10:/tftproot/gentoo_B on /mnt/test/N type nfs (rw,addr=192.168.39.10)
none on /mnt/test/T type tmpfs (rw)
none on /mnt/test/A type unionfs (rw,dirs=/mnt/test/T=rw:/mnt/test/N=ro)

gs test # touch A/x
touch: setting times of `A/x': Operation not permitted

gs test # ls A

bin   dev  home  mnt  proc  sbin          sys  usr
boot  etc  lib   opt  root  stateless.sh  tmp  var

gs test # touch N/x

gs test # ls A

bin   dev  home  mnt  proc  sbin          sys  usr  x
boot  etc  lib   opt  root  stateless.sh  tmp  var

gs test # touch A/x
touch: setting times of `A/x': Operation not permitted gs test #

Any hints?

-
Wilhelm

Quoted 73 lines

Hi!!

Quoted 17 lines

The same problem arises also a simpler setup (on the a single machine):

gs ~ # mount
/dev/hda1 on / type ext3 (rw,noatime)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
udev on /dev type tmpfs (rw,nosuid)
devpts on /dev/pts type devpts (rw)
/dev/hdb1 on /tftproot type ext3 (rw,noatime)
shm on /dev/shm type tmpfs (rw,noexec,nosuid,nodev) nfsd on /proc/fs/nfs type nfsd (rw)
/dev/hdc on /mnt/cdrom type iso9660 (ro)
none on /tftproot/gentoo_B type unionfs (rw,dirs=/tftproot/gentoo_Bdiff=rw:/tftproot/gentoo_A=ro) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) 192.168.39.10:/tftproot/gentoo_B on /mnt/test/N type nfs (rw,addr=192.168.39.10)

If this share is exported RO it should be mounted RO only. Unfortunately I
may mount a RO nfs share with RW option on a client (of course it does not
renders the share writeable) - but it will confuse unionfs. If you mount your share explicitly RO you shouldn't get a permission denied...

We had the same problem with RO nfs root filesystem unioned with RW tempfs
on top of it. Hope I got the idea right what you intend to do :-))

Ciao,
Dirk

Quoted 37 lines

none on /mnt/test/T type tmpfs (rw)
none on /mnt/test/A type unionfs
(rw,dirs=/mnt/test/T=rw:/mnt/test/N=ro)
gs ~ #

As you see 192.168.39.10:/tftproot/gentoo_B is exported and mounted on /mnt/test/N. This is union-mounted together with tmpfs /mnt/test/T on /mnt/test/A.

On /mnt/test/A we get:

gs A # touch x
touch: setting times of `x': Operation not permitted

Am Montag, 5. Dezember 2005 13:48 schrieben Sie:

Quoted 11 lines
Quoted 4 lines

NFS: Buggy server - nlink == 0!
nfs_fhget: iget failed

We are using unionfs-1.0.14

Step 1: Try updating to 1.1.1 or some CVS snapshot.

Jan Engelhardt

--
--
Wilhelm Meier
email: [email protected]
_______________________________________________
unionfs mailing list
[email protected]
http://www.fsl.cs.sunysb.edu/mailman/listinfo/unionfs

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs