NFS-mounted, read-only /dev unusable in 2.6

Hi,

we boot diskless clients using nfsroot. The server exports its own / with "ro,no_root_squash" and the clients mount it via the nfsroot parameter. Then I run my own init script to mount client specific /dev, /var, /etc all rw.

In this scenario, the 2.6.7 kernel fails to open an initial console, since /dev is ro.

The 2.4 kernel was able to open an initial console, and I could also echo sth. explicitely to /dev/console, even when it was still the ro-mounted fs from the server.

With the 2.6.7 kernel, this will fail with "permission denied", and that's why the kernel cannot open an initial console. It will go on silently until I mount the client-specific /dev in my init script and redirect all output.

Similar, using /dev/ram0 for creating an initial ramdisk will fail when / is mounted ro from the server, and server and client use both kernel 2.6.7. If the client runs 2.4, it is possible...

Is that change between 2.4 and 2.6 desired or a bug? It sounds correct that one cannot use a node "/dev/console" for writing if it is mounted read-only from a NFS server, but it was very useful in 2.4.

Or is there any other way to get an initial console or output any messages from an init script if one boots via nfsroot and / (and thus, /dev) is only exported read-only from the server?

I need that to get possible error messages from my own init script to see what fails before I mount the client /dev.

cu,
Frank

Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Frank Steiner <[email protected]> wrote:

| Or is there any other way to get an initial console or
| output any messages from an init script if one boots via nfsroot
| and  / (and thus, /dev) is only exported read-only from the
| server?

You can boot with a ramdisk as root, initialized with an initrd, and then perform all NFS mounts manually in the init script. You can use pivot_root to switch to an NFS root to get rid of the ramdisk.

Dick Streefland                      ////                      Altium BV
[email protected]           (@ @)          http://www.altium.com
--------------------------------oOO--(_)--OOo---------------------------

I'm hoping for an easier solution, because it's a lot of work just to get the console messages onto the screen. But maybe I have to go through this :-)

Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049
* Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *

Maybe I'm missing something, but in the 2.6 series, wasn't the ability to mount subdirectories with different options, also, added? Would it be possible to export and mount /dev with rw options to a specific client?

Or, more radical, if the roots of the clients end up being mounted RW eventually anyway, why not specify 'rw' in the lilo option? It's not like it is a local filesystem that may be corrupt where one should boot from it RO until it is checked...

But I've never tried either approach, so it's really only an idea/possibility.

Hope something works....the booting with a local initrd seems like it would be a pain...

good luck,
Linda

Yes, that's an option. But all this needs to be done in the init script, since with nfsroot I only get /. So mounting /dev rw from the server is similar to figuring out which client-specific /dev should be mounted. But in this init process things can fail and I want to see those messages. If I fail to mount the server /dev rw for any reason, I won't see the error message :-(

No, the real / is exported ro from the server and stays mounted ro on the clients. Only the client specific /dev, /var and /etc are mounted rw...

cu,
Frank

Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049
* Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *

In article <[email protected]>,

If having /dev mounted read-only means you cannot open devices like /dev/console read/write then that is a bug in the NFS client in the kernel.

On all other filesystems (ext2, ext3, xfs etc) there's no problem opening devices r/w on a read-only filesystem.

Mike.

The question is, what is a "manamanap".
The question is, who cares ?

Which matches the fact the it works with server running 2.6 and client running 2.4.

Should I report that as bug to someone special?

cu,
Frank

Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Assuming you have a way to reproduce this, to the NFS client maintainer - see the file MAINTAINERS in the kernel source.

But I just tried to reproduce this on 2.6.7-rc2 (it's what my workstation happens to be running) and I can't. I can mount an nfs-exported /dev from both 2.4 and 2.6 servers read-only and I can open devices on that read-only mount just fine.

Mike.

The question is, what is a "manamanap".
The question is, who cares ?

Thanks! I will do so.

It happens only when the server exports the fs read-only. When it exports the fs rw and you mount it ro on the client, it works. But with exporting ro, it fails.

Before I report this, I guess I should try 2.6.8-rc3 and see if it works there.

Thanks!
cu,
Frank

Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Yes, it was a bug in the NFS server in 2.6 that was fixed fairly recently.

NeilBrown

Hi Neil,

it still occurs in 2.6.8rc3 in some way:

Server and client both running 2.6.8rc3. The client mounts the /dev directory from the server *read-only always*:

- when the server exports its /dev ro, the client cannot echo sth. to the mounted dev/console
- when the server exports its /dev rw, the client can echo to dev/console, although it has mounted it ro.

A client running kernel 2.4 can echo to dev/console in both cases, even with the server exporting /dev ro.

So I'm not sure if this is a client or server issue, but I guess that Trond is reading NFS stuff here, too...

cu,
Frank

Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049
* Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *

.... It looks like it was *so* recent that I haven't actually submitted the patch yet :-(

I think someone reported it, I sent them a patch, and never got a confirmation that it worked (or missed it if I did) so the patch didn't progres..

Try the following.

Thanks,
NeilBrown

### Diffstat output

 ./fs/nfsd/vfs.c |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff ./fs/nfsd/vfs.c~current~ ./fs/nfsd/vfs.c
@@ -376,12 +376,13 @@ static struct accessmap	nfs3_anyaccess[]
 	 * to the server to check for access for things like /dev/null
 	 * (which really, the server doesn't care about).  So
 	 * We provide simple access checking for them, looking
-	 * mainly at mode bits
+	 * mainly at mode bits, and we make sure to ignore read-only
+	 * filesystem checks
 	 */
     {	NFS3_ACCESS_READ,	MAY_READ			},
     {	NFS3_ACCESS_EXECUTE,	MAY_EXEC			},
-    {	NFS3_ACCESS_MODIFY,	MAY_WRITE			},
-    {	NFS3_ACCESS_EXTEND,	MAY_WRITE			},
+    {	NFS3_ACCESS_MODIFY,	MAY_WRITE|MAY_LOCAL_ACCESS	},
+    {	NFS3_ACCESS_EXTEND,	MAY_WRITE|MAY_LOCAL_ACCESS	},
 
     {	0,			0				}
 };

Hi Neil,

it works :-)) Especially, I got my initial console back for the diskless clients and can again see all the debugging output from my init script, and that really really helps.

Thanks a lot for that patch!

cu,
Frank

Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049