Deploying a pNFS / LAYOUT4_BLOCK_VOLUME configuration
Subsystems (2)
Hi,
I'd like to set up from scratch a test bed using pNFS with block devices as I did a few month ago with LAYOUT4_NFSV4_1_FILES. My question about the kernel and nfs-utils versions to be used ? Which one are the right ones ? Where could I get it ?
Regards
Philippe
Quoted 6 lines · DENIEL Philippe
Hi,
I'd like to set up from scratch a test bed using pNFS with block devices as I did a few month ago with
LAYOUT4_NFSV4_1_FILES.
My question about the kernel and nfs-utils versions to be used ? Which one are the right ones ? Where could I get it ?
Hi Philippe,
The instructions are documented here:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Block_Server_Setup_Instructions
The development kernel is the current one, pnfs-all-latest branch. and the nfs utils are here:git://linux-nfs.org/~bhalevy/pnfs-nfs-utils.git
You'll now need the user mode daemon: blkmapd
After make and make install, see man (8) blkmapd
Jim, mind taking a look at the wiki page and update it, in particular anything regarding blkmapd?
Benny
Quoted 8 lines
Regards
Philippe
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Benny Halevy wrote:
Jim, mind taking a look at the wiki page and update it, in particular anything regarding blkmapd?
I can't comment on the server setup. For the client, there are Fedora specific instructions here:
http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup
If you're not using Fedora, you want to install the kernel and nfs-utils from git. I'm working on a wiki page with generic client instructions but it goes something like this:
iscsiadm -m discovery -t sendtargets -p pnfs-server-iscsi -l service blkmapd restart
mount -t nfs4 -o minorversion=1 pnfs-server-1:/pnfs /mnt [ run tests ]
umount /mnt
iscsiadm -m node -U all
Benny Halevy wrote:
Quoted 6 lines · DENIEL Philippe
Hi,
I'd like to set up from scratch a test bed using pNFS with block devices as I did a few month ago with
LAYOUT4_NFSV4_1_FILES.
My question about the kernel and nfs-utils versions to be used ? Which one are the right ones ? Where could I get it ?
Hi Philippe,
The instructions are documented here:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Block_Server_Setup_Instructions
I have added a section on the block layout client to the Fedora client setup wiki page:
http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup
For now the isntructions apply to Fedora, which is what we test with. Eventually I will make this more generic.
Hi Jim,
Quoted 3 lines
The instructions are documented here:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Block_Server_Setup_Instructions
I followed these instructions. Compiling the kernel is OK and so is compilation of pnfs-nfs-utils . I run client and server on F13. They both are virtual machines managed by virt-manager.
I used your start script to make one of my own (I had to change to argument to tgtadm related to the ip name of the iscsi array) but it was ok. The only issue I have is about the ctl program. It stops as soon as it starts with this message:
# ./ctl -u write: File exists
I then ran strace to know more about it and it showed this:
munmap(0x7f5c533ac000, 47451) = 0
*open("/proc/fs/pnfs_block/ctl", O_RDWR) = 3*
*write(3, "\1\0\0\0", 4) = -1 EEXIST (File exists)*
dup(2) = 4
fcntl(4, F_GETFL) = 0x8002 (flags
O_RDWR|O_LARGEFILE)
fstat(4, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5c533b7000
lseek(4, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
write(4, "write: File exists\n", 19write: File exists
) = 19
close(4) = 0 munmap(0x7f5c533b7000, 4096) = 0 exit_group(1) = ?
Apparently writing to /proc/fs/pnfs_block/ctl does not work. Do you have an idea on what I should do to fix this ?
Quoted 5 lines
I have added a section on the block layout client to the Fedora client setup wiki page:
http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup
Is there a layoutdriver to modprobe on the client or is it embedded in the kernel itself ?
Philippe
DENIEL Philippe wrote:
Apparently writing to /proc/fs/pnfs_block/ctl does not work. Do you have an idea on what I should do to fix this ?
I have not worked with the server side but will be in the coming months. Maybe someone on the list can help.
Quoted 4 lines
I have added a section on the block layout client to the Fedora client setup wiki page:
http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup
Is there a layoutdriver to modprobe on the client or is it embedded in the kernel itself ?
You could build it in but I normally build it as a module. The startup script assumes it's a module and probes it:
fedora# grep blocklayoutdriver /etc/init.d/blkmapd
modprobe -q blocklayoutdriver
fedora#
Right now the nfs-utils rpm doesn't install the startup script. I'll fix this but for now just copy it from the source:
cp nfs-utils/utils/blkmapd/etc/initd/initd.redhat /etc/init.d/blkmapd chmod +x /etc/init.d/blkmapd