Skip to contentSkip to conversation

Deploying a pNFS / LAYOUT4_BLOCK_VOLUME configuration

DISCUSSION DENIEL Philippe 3 participants 6 messages Sep 14, 2010 – Sep 15, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

I'd like to set up from scratch a test bed using pNFS with block devices as I did a few month ago with LAYOUT4_NFSV4_1_FILES. My question about the kernel and nfs-utils versions to be used ? Which one are the right ones ? Where could I get it ?

Regards

Philippe

#1 lore raw
Quoted 6 lines · DENIEL Philippe

Hi,

I'd like to set up from scratch a test bed using pNFS with block devices as I did a few month ago with LAYOUT4_NFSV4_1_FILES.
My question about the kernel and nfs-utils versions to be used ? Which one are the right ones ? Where could I get it ?

Hi Philippe,

The instructions are documented here:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Block_Server_Setup_Instructions

The development kernel is the current one, pnfs-all-latest branch. and the nfs utils are here:
git://linux-nfs.org/~bhalevy/pnfs-nfs-utils.git

You'll now need the user mode daemon: blkmapd
After make and make install, see man (8) blkmapd

Jim, mind taking a look at the wiki page and update it, in particular anything regarding blkmapd?

Benny

Quoted 8 lines

Regards

Philippe
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw

Benny Halevy wrote:

Jim, mind taking a look at the wiki page and update it, in particular anything regarding blkmapd?

I can't comment on the server setup. For the client, there are Fedora specific instructions here:

http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup

If you're not using Fedora, you want to install the kernel and nfs-utils from git. I'm working on a wiki page with generic client instructions but it goes something like this:

iscsiadm -m discovery -t sendtargets -p pnfs-server-iscsi -l service blkmapd restart
mount -t nfs4 -o minorversion=1 pnfs-server-1:/pnfs /mnt [ run tests ]
umount /mnt
iscsiadm -m node -U all

#3 lore raw
In reply to Benny Halevy

Benny Halevy wrote:

Quoted 6 lines · DENIEL Philippe

Hi,

I'd like to set up from scratch a test bed using pNFS with block devices as I did a few month ago with LAYOUT4_NFSV4_1_FILES.
My question about the kernel and nfs-utils versions to be used ? Which one are the right ones ? Where could I get it ?

Hi Philippe,

The instructions are documented here:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Block_Server_Setup_Instructions

I have added a section on the block layout client to the Fedora client setup wiki page:

http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup

For now the isntructions apply to Fedora, which is what we test with. Eventually I will make this more generic.

#4 lore raw

Hi Jim,

Quoted 3 lines

The instructions are documented here:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Block_Server_Setup_Instructions

I followed these instructions. Compiling the kernel is OK and so is compilation of pnfs-nfs-utils . I run client and server on F13. They both are virtual machines managed by virt-manager.

I used your start script to make one of my own (I had to change to argument to tgtadm related to the ip name of the iscsi array) but it was ok. The only issue I have is about the ctl program. It stops as soon as it starts with this message:

   # ./ctl -u
   write: File exists

I then ran strace to know more about it and it showed this:

munmap(0x7f5c533ac000, 47451) = 0
*open("/proc/fs/pnfs_block/ctl", O_RDWR) = 3*
*write(3, "\1\0\0\0", 4) = -1 EEXIST (File exists)*
dup(2) = 4
fcntl(4, F_GETFL) = 0x8002 (flags
O_RDWR|O_LARGEFILE)
fstat(4, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5c533b7000
lseek(4, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
write(4, "write: File exists\n", 19write: File exists
) = 19

   close(4)                                = 0
   munmap(0x7f5c533b7000, 4096)            = 0
   exit_group(1)                           = ?

Apparently writing to /proc/fs/pnfs_block/ctl does not work. Do you have an idea on what I should do to fix this ?

Quoted 5 lines

I have added a section on the block layout client to the Fedora client setup wiki page:

http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup

Is there a layoutdriver to modprobe on the client or is it embedded in the kernel itself ?

Philippe

#5 lore raw

DENIEL Philippe wrote:

Apparently writing to /proc/fs/pnfs_block/ctl does not work. Do you have an idea on what I should do to fix this ?

I have not worked with the server side but will be in the coming months. Maybe someone on the list can help.

Quoted 4 lines

I have added a section on the block layout client to the Fedora client setup wiki page:

http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup

Is there a layoutdriver to modprobe on the client or is it embedded in the kernel itself ?

You could build it in but I normally build it as a module. The startup script assumes it's a module and probes it:

fedora# grep blocklayoutdriver /etc/init.d/blkmapd
modprobe -q blocklayoutdriver
fedora#

Right now the nfs-utils rpm doesn't install the startup script. I'll fix this but for now just copy it from the source:

cp nfs-utils/utils/blkmapd/etc/initd/initd.redhat /etc/init.d/blkmapd chmod +x /etc/init.d/blkmapd