Blocklayout

Hi Guys,

i know, no one is using block-layout (it looks like ;D) and i also know that's not actively maintained, but i try to set up a block-layout server together with pnfs.

i found these two wiki pages:

http://wiki.linux-nfs.org/wiki/index.php/PNFS_Block_Server_Setup_Instructions
http://wiki.linux-nfs.org/wiki/index.php/Fedora_pNFS_Client_Setup

till now i did the following:

setup Storage and client described in the howtos (using iscsi-storages).
i used parted to give labels to my devices and so on. Ctl-daemon is running on the server and i got following output:

[root@blkserver sbin]# /root/ctl/ctl -u
[root@blkserver sbin]# Processing: MSG_VERS : Version: 8
Processing: MSG_VERS : Version: 8
Processing: MSG_DMCHK : DM Volume: True
Processing: MSG_DMGET : Vol info: 0 6291456 striped 2 128 8:17 2048 8:33 2048
major 8, minor 17
major 8, minor 33
Processing: MSG_GETSIG : get_sig: 8:17: get_sig: Got sig at: 1, within sector 56, len 16

    42 f6 2e 24 cc ed 69 46   8b cb 0b 82 fa 5a a4 1a   
Processing: MSG_GETSLICE: get_slice: 8:17: Dev: 8:17, Start: 34, Length: 4196285
Processing: MSG_GETSIG  : get_sig: 8:33: get_sig: Got sig at: 1, within sector 56, len 16
    8c 62 64 e7 a7 6b 1a 4f   a3 21 bd d1 f6 d4 a9 a1   

Processing: MSG_GETSLICE: get_slice: 8:33: Dev: 8:33, Start: 34, Length: 4196285

iscsi-targets with the cliet are connected.

on client-side i did the instructions from the howto. But the output from blkmpad looks a little bit curious:

[root@Client2 sbin]# ./blkmapd -f
blkmapd: process_deviceinfo: 5 vols
blkmapd: decode_blk_signature: si_comps[0]: bs_length 16, bs_string ���#�� A��a��0^
blkmapd: read_cmp_blk_sig: /dev/sdb sig ���#�� A��a��0^� at 568
blkmapd: decode_blk_volume: simple 0
blkmapd: decode_blk_volume: slice 1
0�Bhapd: decode_blk_signature: si_comps[0]: bs_length 16, bs_string � L�R�_�FI
0�Bhapd: read_cmp_blk_sig: /dev/sdc sig �
L�R�_�FI� at 568
blkmapd: decode_blk_volume: simple 2
blkmapd: decode_blk_volume: slice 3
blkmapd: process_deviceinfo: 5 vols
blkmapd: decode_blk_signature: si_comps[0]: bs_length 16, bs_string ���#�� A��a��0^
blkmapd: read_cmp_blk_sig: /dev/sdb sig ���#�� A��a��0^� at 568
blkmapd: decode_blk_volume: simple 0
blkmapd: decode_blk_volume: slice 1
0�Bhapd: decode_blk_signature: si_comps[0]: bs_length 16, bs_string �

i can mount and transer files, but no LAYOUTGET-Operation appears in Wireshark. In /proc/self/mountstat are only 0s... at LAYOUTGET, LAYOUTCOMMIT and LAYOUTRETURN.

I used kernel 3.3.0 from git://linux-nfs.org/~bhalevy/linux-pnfs.git, block-layout module is loaded

i attached some wireshark output, maybe someone can help me. It would be appreciated.

Thanks in advance.

Best Regards

Johannes

Johannes Schild wrote:

on client-side i did the instructions from the howto. But the output from blkmpad looks a little bit curious:

Your blkmapd is out of date. You need 0001-remove-pretty_sig.patch from the wiki.

Hi Jim,

thanks for your fast reply. I applied the patch, but didnt changed. I got no Layout information on Wireshark. Only the output of blockmapd changed...

[root@Client2 sbin]# ./blkmapd -f
blkmapd: process_deviceinfo: 5 vols
blkmapd: decode_blk_volume: simple 0
blkmapd: decode_blk_volume: slice 1
blkmapd: decode_blk_volume: simple 2
blkmapd: decode_blk_volume: slice 3
blkmapd: process_deviceinfo: 5 vols
blkmapd: decode_blk_volume: simple 0
blkmapd: decode_blk_volume: slice 1
blkmapd: decode_blk_volume: simple 2
blkmapd: decode_blk_volume: slice 3

Any ideas?

Thanks

Johannes

Hi Guys,

some additional Information,

[ 1644.874549] FS-Cache: Loaded
[ 1644.895122] NFS: Registering the id_resolver key type
[ 1644.895158] FS-Cache: Netfs 'nfs' registered for caching
[ 1644.920204] blkmapd: sending ioctl 2285 to a partition!
[ 1644.920207] blkmapd: sending ioctl 2285 to a partition!
[ 1644.920208] blkmapd: sending ioctl 2285 to a partition!
[ 1644.920263] blkmapd: sending ioctl 2285 to a partition!
[ 1644.920264] blkmapd: sending ioctl 2285 to a partition!
[ 1644.920265] blkmapd: sending ioctl 2285 to a partition!
[ 1653.129460] blkmapd: sending ioctl 2285 to a partition!
[ 1653.129463] blkmapd: sending ioctl 2285 to a partition!
[ 1653.129464] blkmapd: sending ioctl 2285 to a partition!
[ 1653.129528] blkmapd: sending ioctl 2285 to a partition!
[ 1653.129530] blkmapd: sending ioctl 2285 to a partition!
[ 1653.129531] blkmapd: sending ioctl 2285 to a partition!
[ 1653.136682] NFS: set_pnfs_layoutdriver: Error initializing pNFS layout driver 3.

someone any ideas whats going wrong? The above output is from dmesg.
Iam using 3.4.7-1.fc16.x86_64 Kernel from Fedora. The utils are from http://steved.fedorapeople.org/repos/pnfs.

Thanks.

Johannes

DQo+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IGxpbnV4LW5mcy1vd25lckB2 Z2VyLmtlcm5lbC5vcmcgW21haWx0bzpsaW51eC1uZnMtb3duZXJAdmdlci5rZXJuZWwub3JnXSBP biBCZWhhbGYgT2YgSm9oYW5uZXMNCj4gU2NoaWxkDQo+IFNlbnQ6IFRodXJzZGF5LCBBdWd1c3Qg MDksIDIwMTIgNDo0MyBQTQ0KPiBUbzogSm9oYW5uZXMgU2NoaWxkOyByZWVzQHVtaWNoLmVkdQ0K PiBDYzogbGludXgtbmZzQHZnZXIua2VybmVsLm9yZw0KPiBTdWJqZWN0OiBSZTogQmxvY2tsYXlv dXQNCj4gDQo+IEhpIEd1eXMsDQo+IA0KPiBzb21lIGFkZGl0aW9uYWwgSW5mb3JtYXRpb24sDQo+ IA0KPiBbIDE2NDQuODc0NTQ5XSBGUy1DYWNoZTogTG9hZGVkDQo+IFsgMTY0NC44OTUxMjJdIE5G UzogUmVnaXN0ZXJpbmcgdGhlIGlkX3Jlc29sdmVyIGtleSB0eXBlDQo+IFsgMTY0NC44OTUxNThd IEZTLUNhY2hlOiBOZXRmcyAnbmZzJyByZWdpc3RlcmVkIGZvciBjYWNoaW5nDQo+IFsgMTY0NC45 MjAyMDRdIGJsa21hcGQ6IHNlbmRpbmcgaW9jdGwgMjI4NSB0byBhIHBhcnRpdGlvbiENCj4gWyAx NjQ0LjkyMDIwN10gYmxrbWFwZDogc2VuZGluZyBpb2N0bCAyMjg1IHRvIGEgcGFydGl0aW9uIQ0K PiBbIDE2NDQuOTIwMjA4XSBibGttYXBkOiBzZW5kaW5nIGlvY3RsIDIyODUgdG8gYSBwYXJ0aXRp b24hDQo+IFsgMTY0NC45MjAyNjNdIGJsa21hcGQ6IHNlbmRpbmcgaW9jdGwgMjI4NSB0byBhIHBh cnRpdGlvbiENCj4gWyAxNjQ0LjkyMDI2NF0gYmxrbWFwZDogc2VuZGluZyBpb2N0bCAyMjg1IHRv IGEgcGFydGl0aW9uIQ0KPiBbIDE2NDQuOTIwMjY1XSBibGttYXBkOiBzZW5kaW5nIGlvY3RsIDIy ODUgdG8gYSBwYXJ0aXRpb24hDQo+IFsgMTY1My4xMjk0NjBdIGJsa21hcGQ6IHNlbmRpbmcgaW9j dGwgMjI4NSB0byBhIHBhcnRpdGlvbiENCj4gWyAxNjUzLjEyOTQ2M10gYmxrbWFwZDogc2VuZGlu ZyBpb2N0bCAyMjg1IHRvIGEgcGFydGl0aW9uIQ0KPiBbIDE2NTMuMTI5NDY0XSBibGttYXBkOiBz ZW5kaW5nIGlvY3RsIDIyODUgdG8gYSBwYXJ0aXRpb24hDQo+IFsgMTY1My4xMjk1MjhdIGJsa21h cGQ6IHNlbmRpbmcgaW9jdGwgMjI4NSB0byBhIHBhcnRpdGlvbiENCj4gWyAxNjUzLjEyOTUzMF0g YmxrbWFwZDogc2VuZGluZyBpb2N0bCAyMjg1IHRvIGEgcGFydGl0aW9uIQ0KPiBbIDE2NTMuMTI5 NTMxXSBibGttYXBkOiBzZW5kaW5nIGlvY3RsIDIyODUgdG8gYSBwYXJ0aXRpb24hDQo+IFsgMTY1 My4xMzY2ODJdIE5GUzogc2V0X3BuZnNfbGF5b3V0ZHJpdmVyOiBFcnJvciBpbml0aWFsaXppbmcg cE5GUyBsYXlvdXQgZHJpdmVyIDMuDQo+IA0KPiBzb21lb25lIGFueSBpZGVhcyB3aGF0cyBnb2lu ZyB3cm9uZz8gVGhlIGFib3ZlIG91dHB1dCBpcyBmcm9tIGRtZXNnLg0KPiBJYW0gdXNpbmcgMy40 LjctMS5mYzE2Lng4Nl82NCBLZXJuZWwgZnJvbSBGZWRvcmEuIFRoZSB1dGlscyBhcmUgZnJvbQ0K PiBodHRwOi8vc3RldmVkLmZlZG9yYXBlb3BsZS5vcmcvcmVwb3MvcG5mcy4NCj4gDQpUaGVyZSBp cyBhIHJlZ3Jlc3Npb24gYWJvdXQgc2F2aW5nIHByb3BlciBwbmZzX2Jsa3NpemUgYmVmb3JlIHNl dHRpbmcgYmxvY2sgbGF5b3V0IGRyaXZlciBieSBjb21taXQgZTM4ZWI2NTAgKE5GUzogc2V0X3Bu ZnNfbGF5b3V0ZHJpdmVyKCkgZnJvbSBuZnM0X3Byb2NfZnNpbmZvKCkpLg0KSWYgeW91ciBjbGll bnQgaW5jbHVkZXMgdGhlIHNwZWNpZmljIGNvbW1pdCAoYWx0aG91Z2ggSSBkb24ndCB0aGluayB0 aGUgY29tbWl0IGlzIGluIDMuNCBrZXJuZWwpLCBwbGVhc2UgcmV0cnkgYWZ0ZXIgYXBwbHlpbmcg dGhlIGZpeCBhdCBodHRwOi8vd3d3LnNwaW5pY3MubmV0L2xpc3RzL2xpbnV4LW5mcy9tc2czMTg2 OC5odG1sCQ0KDQpJZiB5b3Ugc3RpbGwgc2VlIHNhbWUgZXJyb3IsIHBsZWFzZSBlbmFibGUgY2xp ZW50IGRlYnVnIGJ5DQplY2hvIDY1NTM1ID4gL3Byb2Mvc3lzL3N1bnJwYy9uZnNfZGVidWcNCg0K QW5kIHJlZG8gbW91bnQgYW5kIHNlbmQgdGhlIGxvZy4NCg0KVGhhbmtzLA0KVGFvDQoNCg==

-------- Original-Nachricht --------

The patch doesn't fit my kernel. My cliet.c looks a little bit different from the client.c in the patch:

if (server->wsize > NFS_MAX_FILE_IO_SIZE)

                server->wsize = NFS_MAX_FILE_IO_SIZE;
        server->wpages = (server->wsize + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT;
        server->pnfs_blksize = fsinfo->blksize;
        set_pnfs_layoutdriver(server, mntfh, fsinfo->layouttype);

        server->wtmult = nfs_block_bits(fsinfo->wtmult, NULL);

The same thing on nfs4proc.c:

{
        nfs_fattr_init(fsinfo->fattr);
        return nfs4_do_fsinfo(server, fhandle, fsinfo);
}

there is no error routine or if-statement like in the patch.
that was a different kernel i tried, kernel-3.3.0-0.rc2.git1.1.pnfsd.fc17.src.rpm from http://steved.fedorapeople.org/repos/pnfs

Thats the complete output from dmesg after mounting and try to create a file (i think thats only nfsv4):

[    4.026472] iscsi: registered transport (bnx2i)
[    4.029122] iscsi: registered transport (be2iscsi)
[   81.340654] pcnet32 0000:02:05.0: eth1: link up
[   92.600159] eth1: no IPv6 routers present
[  107.568142] scsi3 : iSCSI Initiator over TCP/IP
[  107.831688] scsi 3:0:0:0: RAID              IET      Controller       0001 PQ: 0 ANSI: 5
[  107.833425] scsi 3:0:0:0: Attached scsi generic sg4 type 12
[  107.836869] scsi 3:0:0:1: Direct-Access     IET      VIRTUAL-DISK     0001 PQ: 0 ANSI: 5
[  107.837494] sd 3:0:0:1: Attached scsi generic sg5 type 0
[  107.838664] sd 3:0:0:1: [sdd] 4196352 512-byte logical blocks: (2.14 GB/2.00 GiB)
[  107.838750] scsi 3:0:0:2: Direct-Access     IET      VIRTUAL-DISK     0001 PQ: 0 ANSI: 5
[  107.838843] sd 3:0:0:2: Attached scsi generic sg6 type 0
[  107.843793] sd 3:0:0:1: [sdd] Write Protect is off
[  107.843796] sd 3:0:0:1: [sdd] Mode Sense: 49 00 00 08
[  107.844193] sd 3:0:0:2: [sde] 4196352 512-byte logical blocks: (2.14 GB/2.00 GiB)
[  107.844814] sd 3:0:0:1: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  107.845943] sd 3:0:0:2: [sde] Write Protect is off
[  107.845945] sd 3:0:0:2: [sde] Mode Sense: 49 00 00 08
[  107.846699] sd 3:0:0:2: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  107.856535]  sdd: sdd1
[  107.859230]  sde: sde1
[  107.859941] sd 3:0:0:1: [sdd] Attached SCSI disk
[  107.861468] sd 3:0:0:2: [sde] Attached SCSI disk
[  179.557893] FS-Cache: Loaded
[  179.563310] NFS: Registering the id_resolver key type
[  179.563339] FS-Cache: Netfs 'nfs' registered for caching
[  207.222318] NFS: set_pnfs_layoutdriver: Error initializing pNFS layout driver 3.

Thanks

Johannes

Then you don't have the buggy commit.

<snip...>

This is the same log without nfs_debug. Please run "echo 65535 > /proc/sys/sunrpc/nfs_debug" and then mount, to get a verbose log output.

Thanks,
Tao

Hi Tao,

sry, here the complete debug output:

[  107.568142] scsi3 : iSCSI Initiator over TCP/IP
[  107.831688] scsi 3:0:0:0: RAID              IET      Controller       0001 PQ: 0 ANSI: 5
[  107.833425] scsi 3:0:0:0: Attached scsi generic sg4 type 12
[  107.836869] scsi 3:0:0:1: Direct-Access     IET      VIRTUAL-DISK     0001 PQ: 0 ANSI: 5
[  107.837494] sd 3:0:0:1: Attached scsi generic sg5 type 0
[  107.838664] sd 3:0:0:1: [sdd] 4196352 512-byte logical blocks: (2.14 GB/2.00 GiB)
[  107.838750] scsi 3:0:0:2: Direct-Access     IET      VIRTUAL-DISK     0001 PQ: 0 ANSI: 5
[  107.838843] sd 3:0:0:2: Attached scsi generic sg6 type 0
[  107.843793] sd 3:0:0:1: [sdd] Write Protect is off
[  107.843796] sd 3:0:0:1: [sdd] Mode Sense: 49 00 00 08
[  107.844193] sd 3:0:0:2: [sde] 4196352 512-byte logical blocks: (2.14 GB/2.00 GiB)
[  107.844814] sd 3:0:0:1: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  107.845943] sd 3:0:0:2: [sde] Write Protect is off
[  107.845945] sd 3:0:0:2: [sde] Mode Sense: 49 00 00 08
[  107.846699] sd 3:0:0:2: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  107.856535]  sdd: sdd1
[  107.859230]  sde: sde1
[  107.859941] sd 3:0:0:1: [sdd] Attached SCSI disk
[  107.861468] sd 3:0:0:2: [sde] Attached SCSI disk
[  179.557893] FS-Cache: Loaded
[  179.563310] NFS: Registering the id_resolver key type
[  179.563339] FS-Cache: Netfs 'nfs' registered for caching
[  207.222318] NFS: set_pnfs_layoutdriver: Error initializing pNFS layout driver 3.
[  308.078197] pcnet32 0000:02:01.0: eth0: link up
[  319.441328] eth0: no IPv6 routers present
[59260.317082] NFS: nfs mount opts='minorversion=1,addr=192.168.0.11,clientaddr=192.168.0.13'
[59260.317088] NFS:   parsing nfs mount option 'minorversion=1'
[59260.317092] NFS:   parsing nfs mount option 'addr=192.168.0.11'
[59260.317108] NFS:   parsing nfs mount option 'clientaddr=192.168.0.13'
[59260.317111] NFS: MNTPATH: '/'
[59260.317112] --> nfs4_try_mount()
[59260.317126] --> nfs4_create_server()
[59260.317134] --> nfs4_init_server()
[59260.317135] --> nfs4_set_client()
[59260.317147] --> nfs_get_client(fedorapnfs2,v4)
[59260.317167] NFS: get client cookie (0xffff88003c733400/0xffff88003d211050)
[59260.318191] --> nfs41_callback_up return 0
[59260.318219] --> nfs_get_client() = ffff88003c733400 [new]
[59260.318220] <-- nfs4_set_client() = 0 [new ffff88003c733400]
[59260.318225] <-- nfs4_init_server() = 0
[59260.318268] --> nfs4_proc_exchange_id
[59260.318813] nfs4_proc_exchange_id: Server Implementation ID: domain: , name: , date: 0,0
[59260.318815] <-- nfs4_proc_exchange_id status= 0
[59260.318816] --> nfs4_proc_create_session clp=ffff88003c733400 session=ffff88003c733c00
[59260.318818] nfs4_init_channel_attrs: Fore Channel : max_rqst_sz=1049620 max_resp_sz=1049480 max_ops=8 max_reqs=16
[59260.318820] nfs4_init_channel_attrs: Back Channel : max_rqst_sz=4096 max_resp_sz=4096 max_resp_sz_cached=0 max_ops=2 max_reqs=1
[59260.319156] --> nfs4_setup_session_slot_tables
[59260.319170] --> nfs4_realloc_slot_table: max_reqs=16, tbl->max_slots 0
[59260.319172] nfs4_realloc_slot_table: tbl=ffff88003c733c40 slots=ffff88003681c480 max_slots=16
[59260.319173] <-- nfs4_realloc_slot_table: return 0
[59260.319174] --> nfs4_realloc_slot_table: max_reqs=1, tbl->max_slots 0
[59260.319176] nfs4_realloc_slot_table: tbl=ffff88003c733d80 slots=ffff88003a3d9b68 max_slots=1
[59260.319177] <-- nfs4_realloc_slot_table: return 0
[59260.319187] slot table setup returned 0
[59260.319188] nfs4_proc_create_session client>seqid 2 sessionid 1344524732:4:4:0
[59260.319190] <-- nfs4_proc_create_session
[59260.319190] --> nfs4_proc_get_lease_time
[59260.319192] --> nfs4_get_lease_time_prepare
[59260.319193] --> nfs41_setup_sequence
[59260.319194] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.319196] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.319197] <-- nfs41_setup_sequence slotid=0 seqid=1
[59260.319198] <-- nfs4_get_lease_time_prepare
[59260.319359] encode_sequence: sessionid=1344524732:4:4:0 seqid=1 slotid=0 max_slotid=0 cache_this=0
[59260.319364] Invoking bc_svc_process()
[59260.319519] bc_svc_process() returned w/ error code= 0
[59260.321000] decode_attr_lease_time: file size=90
[59260.321001] decode_attr_maxfilesize: maxfilesize=0
[59260.321002] decode_attr_maxread: maxread=1024
[59260.321003] decode_attr_maxwrite: maxwrite=1024
[59260.321004] decode_attr_time_delta: time_delta=0 0
[59260.321005] decode_attr_pnfstype: bitmap is 0
[59260.321006] decode_attr_layout_blksize: bitmap is 0
[59260.321007] decode_fsinfo: xdr returned 0!
[59260.321008] --> nfs4_get_lease_time_done
[59260.321009] nfs41_sequence_done: Error 0 free the slot 
[59260.321011] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.321011] nfs4_check_drain_fc_complete COMPLETE: Session Fore Channel Drained
[59260.321013] <-- nfs4_get_lease_time_done
[59260.321015] <-- nfs4_proc_get_lease_time return 0
[59260.321017] nfs4_schedule_state_renewal: requeueing work. Lease period = 60
[59260.321034] --> nfs41_proc_reclaim_complete
[59260.321039] --> nfs41_setup_sequence
[59260.321041] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.321042] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.321043] <-- nfs41_setup_sequence slotid=0 seqid=2
[59260.321047] encode_sequence: sessionid=1344524732:4:4:0 seqid=2 slotid=0 max_slotid=0 cache_this=0
[59260.322154] --> nfs4_reclaim_complete_done
[59260.322163] nfs41_sequence_done: Error 0 free the slot 
[59260.322165] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.322166] nfs4_check_drain_fc_complete COMPLETE: Session Fore Channel Drained
[59260.322167] <-- nfs4_reclaim_complete_done
[59260.322175] --> nfs_put_client({2})
[59260.322178] --> nfs4_get_rootfh()
[59260.322184] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.322185] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.322186] --> nfs41_setup_sequence
[59260.322188] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.322189] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.322190] <-- nfs41_setup_sequence slotid=0 seqid=3
[59260.322191] <-- nfs4_setup_sequence status=0
[59260.322195] encode_sequence: sessionid=1344524732:4:4:0 seqid=3 slotid=0 max_slotid=0 cache_this=0
[59260.322587] decode_attr_type: type=040000
[59260.322589] decode_attr_change: change attribute=13246251234597893889
[59260.322590] decode_attr_size: file size=4096
[59260.322591] decode_attr_fsid: fsid=(0x0/0x0)
[59260.322592] decode_attr_fileid: fileid=2
[59260.322593] decode_attr_fs_locations: fs_locations done, error = 0
[59260.322594] decode_attr_mode: file mode=0755
[59260.322595] decode_attr_nlink: nlink=3
[59260.322596] decode_attr_owner: uid=0
[59260.322597] decode_attr_group: gid=0
[59260.322598] decode_attr_rdev: rdev=(0x0:0x0)
[59260.322599] decode_attr_space_used: space used=4096
[59260.322611] decode_attr_time_access: atime=1344525504
[59260.322612] decode_attr_time_metadata: ctime=1344525495
[59260.322613] decode_attr_time_modify: mtime=1344525495
[59260.322614] decode_attr_mounted_on_fileid: fileid=0
[59260.322614] decode_getfattr_attrs: xdr returned 0
[59260.322615] decode_getfattr_generic: xdr returned 0
[59260.322635] nfs41_sequence_done: Error 0 free the slot 
[59260.322636] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.322640] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.322641] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.322642] --> nfs41_setup_sequence
[59260.322643] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.322645] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.322646] <-- nfs41_setup_sequence slotid=0 seqid=4
[59260.322646] <-- nfs4_setup_sequence status=0
[59260.322649] encode_sequence: sessionid=1344524732:4:4:0 seqid=4 slotid=0 max_slotid=0 cache_this=0
[59260.323028] decode_attr_supported: bitmask=fdffbfff:40f9be3e:00000803
[59260.323030] decode_attr_fh_expire_type: expire type=0x0
[59260.323031] decode_attr_link_support: link support=true
[59260.323032] decode_attr_symlink_support: symlink support=true
[59260.323033] decode_attr_aclsupport: ACLs supported=3
[59260.323033] decode_server_caps: xdr returned 0!
[59260.323034] nfs41_sequence_done: Error 0 free the slot 
[59260.323035] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.323041] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.323043] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.323044] --> nfs41_setup_sequence
[59260.323045] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.323046] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.323047] <-- nfs41_setup_sequence slotid=0 seqid=5
[59260.323048] <-- nfs4_setup_sequence status=0
[59260.323051] encode_sequence: sessionid=1344524732:4:4:0 seqid=5 slotid=0 max_slotid=0 cache_this=0
[59260.323438] decode_attr_lease_time: file size=90
[59260.323440] decode_attr_maxfilesize: maxfilesize=18446744073709551615
[59260.323441] decode_attr_maxread: maxread=131072
[59260.323442] decode_attr_maxwrite: maxwrite=131072
[59260.323443] decode_attr_time_delta: time_delta=1 0
[59260.323444] decode_attr_pnfstype: bitmap is 40000000
[59260.323445] decode_attr_layout_blksize: bitmap is 2
[59260.323446] decode_fsinfo: xdr returned 0!
[59260.323447] nfs41_sequence_done: Error 0 free the slot 
[59260.323448] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.323451] <-- nfs4_get_rootfh() = 0
[59260.323452] Server FSID: 0:0
[59260.323452] Mount FH: 8
[59260.323453] --> nfs_probe_fsinfo()
[59260.323454] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.323456] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.323457] --> nfs41_setup_sequence
[59260.323458] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.323459] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.323460] <-- nfs41_setup_sequence slotid=0 seqid=6
[59260.323461] <-- nfs4_setup_sequence status=0
[59260.323463] encode_sequence: sessionid=1344524732:4:4:0 seqid=6 slotid=0 max_slotid=0 cache_this=0
[59260.323735] decode_attr_supported: bitmask=fdffbfff:40f9be3e:00000803
[59260.323736] decode_attr_fh_expire_type: expire type=0x0
[59260.323737] decode_attr_link_support: link support=true
[59260.323738] decode_attr_symlink_support: symlink support=true
[59260.323739] decode_attr_aclsupport: ACLs supported=3
[59260.323740] decode_server_caps: xdr returned 0!
[59260.323741] nfs41_sequence_done: Error 0 free the slot 
[59260.323742] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.323745] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.323747] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.323748] --> nfs41_setup_sequence
[59260.323749] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.323750] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.323777] <-- nfs41_setup_sequence slotid=0 seqid=7
[59260.323777] <-- nfs4_setup_sequence status=0
[59260.323781] encode_sequence: sessionid=1344524732:4:4:0 seqid=7 slotid=0 max_slotid=0 cache_this=0
[59260.324170] decode_attr_lease_time: file size=90
[59260.324171] decode_attr_maxfilesize: maxfilesize=18446744073709551615
[59260.324172] decode_attr_maxread: maxread=131072
[59260.324173] decode_attr_maxwrite: maxwrite=131072
[59260.324174] decode_attr_time_delta: time_delta=1 0
[59260.324175] decode_attr_pnfstype: bitmap is 40000000
[59260.324175] decode_attr_layout_blksize: bitmap is 2
[59260.324176] decode_fsinfo: xdr returned 0!
[59260.324177] nfs41_sequence_done: Error 0 free the slot 
[59260.324178] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.324195] find_pnfs_driver_locked: Searching for id 3, found ffffffffa0409000
[59260.324218] bl_set_layoutdriver enter
[59260.324220] --> _nfs4_getdevicelist
[59260.324222] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.324223] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.324224] --> nfs41_setup_sequence
[59260.324225] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.324226] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.324227] <-- nfs41_setup_sequence slotid=0 seqid=8
[59260.324228] <-- nfs4_setup_sequence status=0
[59260.324231] encode_sequence: sessionid=1344524732:4:4:0 seqid=8 slotid=0 max_slotid=0 cache_this=0
[59260.324598] encoding getdevicelist!
[59260.324600] decode_getdevicelist: num_dev 1
[59260.324601] nfs41_sequence_done: Error 0 free the slot 
[59260.324602] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.324605] <-- _nfs4_getdevicelist status=0
[59260.324606] nfs4_proc_getdevicelist: err=0, num_devs=1
[59260.324607] bl_set_layoutdriver GETDEVICELIST numdevs=1, eof=1
[59260.324608] nfs4_blk_get_deviceinfo max_resp_sz 135168 max_pages 33
[59260.324629] nfs4_blk_get_deviceinfo: dev_id: 
[59260.324630] --> _nfs4_proc_getdeviceinfo
[59260.324632] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.324633] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.324634] --> nfs41_setup_sequence
[59260.324635] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.324636] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.324637] <-- nfs41_setup_sequence slotid=0 seqid=9
[59260.324638] <-- nfs4_setup_sequence status=0
[59260.324642] encode_sequence: sessionid=1344524732:4:4:0 seqid=9 slotid=0 max_slotid=0 cache_this=0
[59260.333545] nfs41_sequence_done: Error 0 free the slot 
[59260.333547] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.333553] <-- _nfs4_proc_getdeviceinfo status=0
[59260.333555] nfs4_blk_get_deviceinfo getdevice info returns 0
[59260.333572] nfs4_blk_decode_device CREATING PIPEFS MESSAGE
[59260.333573] nfs4_blk_decode_device: deviceid: , mincount: 136
[59260.333574] nfs4_blk_decode_device CALLING USERSPACE DAEMON
[59260.333595] NFS: set_pnfs_layoutdriver: Error initializing pNFS layout driver 3.
[59260.333618] set_pnfs_layoutdriver: Using NFSv4 I/O
[59260.333623] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.333624] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.333626] --> nfs41_setup_sequence
[59260.333627] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.333629] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.333630] <-- nfs41_setup_sequence slotid=0 seqid=10
[59260.333631] <-- nfs4_setup_sequence status=0
[59260.333637] encode_sequence: sessionid=1344524732:4:4:0 seqid=10 slotid=0 max_slotid=0 cache_this=0
[59260.333968] decode_attr_maxlink: maxlink=255
[59260.333969] decode_attr_maxname: maxname=255
[59260.333982] decode_pathconf: xdr returned 0!
[59260.333984] nfs41_sequence_done: Error 0 free the slot 
[59260.333985] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.333988] <-- nfs_probe_fsinfo() = 0
[59260.333990] <-- nfs4_create_server() = ffff88003c8d7800
[59260.334116] NFS: get superblock cookie (0xffff88003c8d7800/0xffff88003d2110a0)
[59260.334118] --> nfs4_get_root()
[59260.334120] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.334121] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.334122] --> nfs41_setup_sequence
[59260.334123] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.334125] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.334126] <-- nfs41_setup_sequence slotid=0 seqid=11
[59260.334127] <-- nfs4_setup_sequence status=0
[59260.334130] encode_sequence: sessionid=1344524732:4:4:0 seqid=11 slotid=0 max_slotid=0 cache_this=0
[59260.334427] decode_attr_supported: bitmask=fdffbfff:40f9be3e:00000803
[59260.334428] decode_attr_fh_expire_type: expire type=0x0
[59260.334429] decode_attr_link_support: link support=true
[59260.334430] decode_attr_symlink_support: symlink support=true
[59260.334431] decode_attr_aclsupport: ACLs supported=3
[59260.334432] decode_server_caps: xdr returned 0!
[59260.334433] nfs41_sequence_done: Error 0 free the slot 
[59260.334434] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.334442] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.334444] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.334445] --> nfs41_setup_sequence
[59260.334446] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.334447] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.334448] <-- nfs41_setup_sequence slotid=0 seqid=12
[59260.334449] <-- nfs4_setup_sequence status=0
[59260.334452] encode_sequence: sessionid=1344524732:4:4:0 seqid=12 slotid=0 max_slotid=0 cache_this=0
[59260.334687] decode_attr_type: type=040000
[59260.334688] decode_attr_change: change attribute=13246251234597893889
[59260.334690] decode_attr_size: file size=4096
[59260.334690] decode_attr_fsid: fsid=(0x0/0x0)
[59260.334691] decode_attr_fileid: fileid=2
[59260.334692] decode_attr_fs_locations: fs_locations done, error = 0
[59260.334693] decode_attr_mode: file mode=0755
[59260.334694] decode_attr_nlink: nlink=3
[59260.334696] decode_attr_owner: uid=0
[59260.334697] decode_attr_group: gid=0
[59260.334698] decode_attr_rdev: rdev=(0x0:0x0)
[59260.334699] decode_attr_space_used: space used=4096
[59260.334700] decode_attr_time_access: atime=1344525504
[59260.334701] decode_attr_time_metadata: ctime=1344525495
[59260.334701] decode_attr_time_modify: mtime=1344525495
[59260.334702] decode_attr_mounted_on_fileid: fileid=0
[59260.334703] decode_getfattr_attrs: xdr returned 0
[59260.334704] decode_getfattr_generic: xdr returned 0
[59260.334705] nfs41_sequence_done: Error 0 free the slot 
[59260.334706] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.334725] NFS: nfs_fhget(0:34/2 fh_crc=0x62d40c52 ct=1)
[59260.334727] <-- nfs4_get_root()
[59260.334734] --> nfs41_call_sync_prepare data->seq_server ffff88003c8d7800
[59260.334735] --> nfs4_setup_sequence clp ffff88003c733400 session ffff88003c733c00 sr_slot -1
[59260.334736] --> nfs41_setup_sequence
[59260.334737] --> nfs4_find_slot used_slots=0000 highest_used=4294967295 max_slots=16
[59260.334738] <-- nfs4_find_slot used_slots=0001 highest_used=0 slotid=0 
[59260.334739] <-- nfs41_setup_sequence slotid=0 seqid=13
[59260.334740] <-- nfs4_setup_sequence status=0
[59260.334743] encode_sequence: sessionid=1344524732:4:4:0 seqid=13 slotid=0 max_slotid=0 cache_this=0
[59260.334965] decode_attr_type: type=00
[59260.334966] decode_attr_change: change attribute=13246251234597893889
[59260.334977] decode_attr_size: file size=4096
[59260.334978] decode_attr_fsid: fsid=(0x0/0x0)
[59260.334979] decode_attr_fileid: fileid=0
[59260.334980] decode_attr_fs_locations: fs_locations done, error = 0
[59260.334981] decode_attr_mode: file mode=00
[59260.334982] decode_attr_nlink: nlink=1
[59260.334983] decode_attr_owner: uid=-2
[59260.334983] decode_attr_group: gid=-2
[59260.334984] decode_attr_rdev: rdev=(0x0:0x0)
[59260.334985] decode_attr_space_used: space used=0
[59260.334986] decode_attr_time_access: atime=0
[59260.334987] decode_attr_time_metadata: ctime=1344525495
[59260.334988] decode_attr_time_modify: mtime=1344525495
[59260.334988] decode_attr_mounted_on_fileid: fileid=0
[59260.334989] decode_getfattr_attrs: xdr returned 0
[59260.334990] decode_getfattr_generic: xdr returned 0
[59260.334991] nfs41_sequence_done: Error 0 free the slot 
[59260.334992] nfs4_free_slot: slotid 16 highest_used_slotid -1
[59260.334997] NFS: nfs_update_inode(0:34/2 fh_crc=0x62d40c52 ct=2 info=0x26040)
[59260.334999] NFS: permission(0:34/2), mask=0x1, res=0
[59260.335002] NFS: nfs_lookup_revalidate(/) is valid
[59260.335009] <-- nfs4_try_mount() = 0
[59260.335011] <-- nfs4_mount() = 0

[root@client3 Downloads]#

-------- Original-Nachricht --------

Thanks
Johannes

The deviceid field returned by server is empty. That's why device discovery failed.

Here shows the same error.

You may check server code to see if there is something wrong with getdevicelist proc handling.

Cheers,
Tao

Hi Tao,

Is the deviceid the field where the signature, created with parted, should be stored?

I am not experienced enough to check this i think. Is there other server code than "ctl" (on the howto)?

Thanks

Johannes

For client, deviceid is just a symbol that client uses to ask server about disk structure information. It is server's implementation decision how to construct a deviceid.

I'm not sure about current pnfs block server status, nor if the wiki howto is out of date...

Cheers,
Tao

Hi Tao,

the output von the server (ctl) looks for *me* good...

http://www.mozilla.org/en-US/firefox/update/

[root@fedorapnfs2 sbin]# /root/ctl/ctl -u
[root@fedorapnfs2 sbin]# Processing: MSG_VERS : Version: 8
Processing: MSG_VERS : Version: 8
Processing: MSG_DMCHK : DM Volume: True
Processing: MSG_DMGET : Vol info: 0 6291456 striped 2 128 8:17 2048 8:33 2048
major 8, minor 17
major 8, minor 33
Processing: MSG_GETSIG : get_sig: 8:17: get_sig: Got sig at: 1, within sector 56, len 16

    e9 91 e5 1c d7 cb 20 41   ad d1 61 f9 a7 30 5e fb   
Processing: MSG_GETSLICE: get_slice: 8:17: Dev: 8:17, Start: 34, Length: 4196285
Processing: MSG_GETSIG  : get_sig: 8:33: get_sig: Got sig at: 1, within sector 56, len 16
    9b 0d 30 b8 42 68 0a 4c   95 52 ed 5f d7 46 49 ed   

Processing: MSG_GETSLICE: get_slice: 8:33: Dev: 8:33, Start: 34, Length: 4196285

-------- Original-Nachricht --------

Johannes Schild wrote:

Hi Tao,

the output von the server (ctl) looks for *me* good...

http://www.mozilla.org/en-US/firefox/update/

[root@fedorapnfs2 sbin]# /root/ctl/ctl -u
[root@fedorapnfs2 sbin]# Processing: MSG_VERS : Version: 8

  Processing: MSG_VERS    : Version: 8
  Processing: MSG_DMCHK   : DM Volume: True
  Processing: MSG_DMGET   : Vol info: 0 6291456 striped 2 128 8:17 2048 8:33 2048
      major 8, minor 17
      major 8, minor 33
  Processing: MSG_GETSIG  : get_sig: 8:17: get_sig: Got sig at: 1, within sector 56, len 16
      e9 91 e5 1c d7 cb 20 41   ad d1 61 f9 a7 30 5e fb   
  Processing: MSG_GETSLICE: get_slice: 8:17: Dev: 8:17, Start: 34, Length: 4196285
  Processing: MSG_GETSIG  : get_sig: 8:33: get_sig: Got sig at: 1, within sector 56, len 16
      9b 0d 30 b8 42 68 0a 4c   95 52 ed 5f d7 46 49 ed   

Processing: MSG_GETSLICE: get_slice: 8:33: Dev: 8:33, Start: 34, Length: 4196285

I'm not familiar with this server, but all I see here is slices from two disks. The blkmapd output shows the same thing. Normally these would be striped or catenated to form the file system device. What device topology do you think you're exporting?

Hi Jim,

thanks for trying to help me:

I use LVM and iSCSI to export my device:
LVM for Striping (because ctl needs it), iSCSI for transport.

i created a volume "testserver":

[root@fedorapnfs2 server]# pvs

  PV         VG              Fmt  Attr PSize  PFree  
  /dev/sda3  vg_createfedora lvm2 a--  19.50g      0 
  /dev/sdb1  testserver      lvm2 a--   2.00g 508.00m
  /dev/sdc1  testserver      lvm2 a--   2.00g 508.00m

[root@fedorapnfs2 server]# dmsetup ls --target striped
testserver-share (253, 2)

[root@fedorapnfs2 server]# dmsetup table
vg_createfedora-lv_swap: 0 4128768 linear 8:3 2048
vg_createfedora-lv_root: 0 36765696 linear 8:3 4130816
testserver-share: 0 6291456 striped 2 128 8:17 2048 8:33 2048

The following output is from my iSCSI-targets which i export:

[root@fedorapnfs2 server]# tgtadm --lld iscsi --mode target --op show Target 1: iqn.2012-04.fedorapnfs2:dev01
System information:
Driver: iscsi
State: ready
I_T nexus information:
I_T nexus: 2
Initiator: iqn.1994-05.com.redhat:f1b658ea7af8
Connection: 0
IP Address: 192.168.0.13
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 2149 MB, Block size: 512
Online: Yes
Removable media: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/sdb
Backing store flags:
LUN: 2
Type: disk
SCSI ID: IET 00010002
SCSI SN: beaf12
Size: 2149 MB, Block size: 512
Online: Yes
Removable media: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/sdc
Backing store flags:
Account information:
ACL information:
ALL

Thanks for your help.

Johannes

Hi Guys,

sorry for bothering you again. But any ideas how to get rid of this error with an empty device id?

Thanks

Johannes

I never used that server before. So don't know if your setup is correct nor whether the code that you use is working itself... Maybe you should ask Benny which code is working first.

Cheers,
Tao