Questions about Exofs

Hi,

i try to set up the object-based layout with pnfs.
I am using the OSC-OSD as target. It works well and i can login.
Trying the scripts "do-*" from git://git.open-osd.org/open-osd.git the do-osd script works for me. The do-exofs doesnt, here i have some question:

1. Is the UUID in the do-exofs script optional?

2. If no how can i get it for my device?

3. It is mandatory to use raid-driver (raid456) in the script? Why?

The error is:
mount -t exofs -o osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000,_netdev /dev/osd0 /mnt/osd0/
mount: wrong fs type, bad option, bad superblock on /dev/osd0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so

dmesg says:
exofs: Unable to mount exofs on (null) pid=0x0 err=-22

Thanks for answering my questions.

Regards
Johannes

Datum: Tue, 15 May 2012 12:48:08 +0300
Von: Boaz Harrosh <[email protected]>
An: Johannes Schild <[email protected]>
CC: [email protected], open-osd <[email protected]>
Betreff: Re: Questions about Exofs

Quoted 5 lines · Johannes Schild

Hi,

i try to set up the object-based layout with pnfs.
I am using the OSC-OSD as target. It works well and i can login. Trying the scripts "do-*" from git://git.open-osd.org/open-osd.git the

do-osd script works for me. The do-exofs doesnt, here i have some question:

Quoted 3 lines

1. Is the UUID in the do-exofs script optional?

No it is not optional

Quoted 2 lines

2. If no how can i get it for my device?

Exactly! the ./do-exofs format command will set this for you, as well as mkfs.exofs the
FS for you.

Quoted 2 lines

3. It is mandatory to use raid-driver (raid456) in the script? Why?

What? where? Rrrr you are right. this is a very old tree. Let me see if I have something newer to push. It should all load automatically now.

But yes the dependency on raid456.ko is built in. Though if you use raid=0 on the format command line it will not be used in run time.

Quoted 3 lines

The error is:
mount -t exofs -o

osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000,_netdev /dev/osd0 /mnt/osd0/

Quoted 8 lines

mount: wrong fs type, bad option, bad superblock on /dev/osd0,
missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so

dmesg says:
exofs: Unable to mount exofs on (null) pid=0x0 err=-22

Probably you forgot the "./do-exofs format" stage. Please read the script before, you might need to edit it. It is the stage that calls mkfs.exofs to make a new filesystem for you. (An OSD is a raw device that can host many filesystems each in it's own osd-partition.

Quoted 7 lines

Thanks for answering my questions.

Regards
Johannes

Cheers
Boaz

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi,

thanks for your fast reply.

But now i run into a nullpointer exception while mounting /dev/osd0

[  329.919112] Modules linked in: raid456 async_raid6_recov async_pq raid6_pq async_memcpy bnx2i cnic cxgb4i cxgb3i iscsi_tcp be2iscsi iscsi_boot_sysfs uio cxgb4 libcxgbi cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi fuse ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack 8021q ip6table_filter ip6_tables garp stp llc fcoe libfcoe libfc scsi_transport_fc scsi_tgt snd_ens1371 gameport snd_rawmidi snd_ac97_codec ac97_bus joydev snd_seq snd_seq_device snd_pcm ppdev vmw_balloon floppy parport_pc snd_timer snd pcnet32 microcode parport pcspkr mii snd_page_alloc i2c_piix4 i2c_core shpchp uinput mptspi mptscsih mptbase scsi_transport_spi [last unloaded: iscsi_tcp] [  329.919133]
[  329.919134] Pid: 2234, comm: mount Not tainted 3.3.0-pnfs #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
[  329.919137] RIP: 0010:[<ffffffff81259799>]  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919139] RSP: 0018:ffff88003c4e5bc8  EFLAGS: 00010246
[  329.919141] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000001795
[  329.919142] RDX: 0000000000000000 RSI: 0000000000000046 RDI: ffff88003c4f5000
[  329.919143] RBP: ffff88003c4e5bd8 R08: 0000000000000000 R09: 0000000000000000
[  329.919144] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88003c4f5000
[  329.919145] R13: ffff88003c4e5d58 R14: 00000000fffffffb R15: ffff88003bd4e360
[  329.919146] FS:  00007f68ce81d800(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
[  329.919148] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  329.919149] CR2: 0000000000000000 CR3: 000000003cbfd000 CR4: 00000000000406f0
[  329.919152] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  329.919155] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  329.919156] Process mount (pid: 2234, threadinfo ffff88003c4e4000, task ffff88003aa42e40)
[  329.919157] Stack:
[  329.919158]  ffff88003c4f5000 ffff88003c4f5800 ffff88003c4e5d18 ffffffff8125a780
[  329.919160]  ffff88003c4e5c84 0000000081f27618 ffffffff81f27600 ffff88003f80c288
[  329.919162]  ffff88003ddbeee0 0000000000000000 ffff88003c4e5c48 ffffffff812c8433
[  329.919164] Call Trace:
[  329.919166]  [<ffffffff8125a780>] exofs_fill_super+0x440/0x6e0
[  329.919171]  [<ffffffff812c8433>] ? idr_pre_get+0x53/0x90
[  329.919173]  [<ffffffff8125a340>] ? exofs_read_lookup_dev_table+0x480/0x480
[  329.919176]  [<ffffffff81180e18>] mount_nodev+0x58/0xb0
[  329.919178]  [<ffffffff812594ff>] exofs_mount+0x4f/0x80
[  329.919179]  [<ffffffff81181e13>] mount_fs+0x43/0x1b0
[  329.919182]  [<ffffffff8119b56a>] vfs_kern_mount+0x6a/0xf0
[  329.919183]  [<ffffffff8119bd44>] do_kern_mount+0x54/0x110
[  329.919185]  [<ffffffff8119d4b4>] do_mount+0x1a4/0x830
[  329.919188]  [<ffffffff8115da33>] ? alloc_pages_current+0xa3/0x110
[  329.919191]  [<ffffffff811229e4>] ? __get_free_pages+0x14/0x50
[  329.919193]  [<ffffffff8119d18a>] ? copy_mount_options+0x3a/0x170
[  329.919194]  [<ffffffff8119dc80>] sys_mount+0x90/0xe0
[  329.919199]  [<ffffffff81605aa9>] system_call_fastpath+0x16/0x1b
[  329.919200] Code: 66 90 41 8b 84 24 70 03 00 00 83 eb 01 39 c3 72 50 41 8b 94 24 74 03 00 00 01 c2 39 d3 73 42 89 da 29 c2 49 8b 84 24 88 03 00 00 <48> 8b 04 d0 48 8b 38 48 85 ff 74 0c 48 c7 00 00 00 00 00 e8 6f
[  329.919214] RIP  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919216]  RSP <ffff88003c4e5bc8>
[  329.919217] CR2: 0000000000000000
[  329.919220] ---[ end trace e5a3b1124e42d9e3 ]---

Is there any possibility to avoid these failure?

Just now i am using the 3.3.0 kernel from the linux-pnfs repository.

Thanks in adavance

Cheers
Johannes

-------- Original-Nachricht --------

Quoted 67 lines

Datum: Tue, 15 May 2012 12:48:08 +0300
Von: Boaz Harrosh <[email protected]>
An: Johannes Schild <[email protected]>
CC: [email protected], open-osd <[email protected]>
Betreff: Re: Questions about Exofs

Quoted 5 lines · Johannes Schild

Hi,

i try to set up the object-based layout with pnfs.
I am using the OSC-OSD as target. It works well and i can login. Trying the scripts "do-*" from git://git.open-osd.org/open-osd.git the

do-osd script works for me. The do-exofs doesnt, here i have some question:

Quoted 3 lines

1. Is the UUID in the do-exofs script optional?

No it is not optional

Quoted 2 lines

2. If no how can i get it for my device?

Exactly! the ./do-exofs format command will set this for you, as well as mkfs.exofs the
FS for you.

Quoted 2 lines

3. It is mandatory to use raid-driver (raid456) in the script? Why?

What? where? Rrrr you are right. this is a very old tree. Let me see if I have something newer to push. It should all load automatically now.

But yes the dependency on raid456.ko is built in. Though if you use raid=0 on the format command line it will not be used in run time.

Quoted 3 lines

The error is:
mount -t exofs -o

osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000,_netdev /dev/osd0 /mnt/osd0/

Quoted 8 lines

mount: wrong fs type, bad option, bad superblock on /dev/osd0, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail  or so

dmesg says:
exofs: Unable to mount exofs on (null) pid=0x0 err=-22

Probably you forgot the "./do-exofs format" stage. Please read the script before, you might need to edit it. It is the stage that calls mkfs.exofs to make a new filesystem for you. (An OSD is a raw device that can host many filesystems each in it's own osd-partition.

Quoted 7 lines

Thanks for answering my questions.

Regards
Johannes

Quoted 7 lines

Cheers
Boaz

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone!
Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Quoted 135 lines · Johannes Schild

Hi,

thanks for your fast reply.

But now i run into a nullpointer exception while mounting /dev/osd0

[  329.919112] Modules linked in: raid456 async_raid6_recov async_pq raid6_pq async_memcpy bnx2i cnic cxgb4i cxgb3i iscsi_tcp be2iscsi iscsi_boot_sysfs uio cxgb4 libcxgbi cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi fuse ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack 8021q ip6table_filter ip6_tables garp stp llc fcoe libfcoe libfc scsi_transport_fc scsi_tgt snd_ens1371 gameport snd_rawmidi snd_ac97_codec ac97_bus joydev snd_seq snd_seq_device snd_pcm ppdev vmw_balloon floppy parport_pc snd_timer snd pcnet32 microcode parport pcspkr mii snd_page_alloc i2c_piix4 i2c_core shpchp uinput mptspi mptscsih mptbase scsi_transport_spi [last unloaded: iscsi_tcp]
[  329.919133]
[  329.919134] Pid: 2234, comm: mount Not tainted 3.3.0-pnfs #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
[  329.919137] RIP: 0010:[<ffffffff81259799>]  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919139] RSP: 0018:ffff88003c4e5bc8  EFLAGS: 00010246
[  329.919141] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000001795
[  329.919142] RDX: 0000000000000000 RSI: 0000000000000046 RDI: ffff88003c4f5000
[  329.919143] RBP: ffff88003c4e5bd8 R08: 0000000000000000 R09: 0000000000000000
[  329.919144] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88003c4f5000
[  329.919145] R13: ffff88003c4e5d58 R14: 00000000fffffffb R15: ffff88003bd4e360
[  329.919146] FS:  00007f68ce81d800(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
[  329.919148] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  329.919149] CR2: 0000000000000000 CR3: 000000003cbfd000 CR4: 00000000000406f0
[  329.919152] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  329.919155] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  329.919156] Process mount (pid: 2234, threadinfo ffff88003c4e4000, task ffff88003aa42e40)
[  329.919157] Stack:
[  329.919158]  ffff88003c4f5000 ffff88003c4f5800 ffff88003c4e5d18 ffffffff8125a780
[  329.919160]  ffff88003c4e5c84 0000000081f27618 ffffffff81f27600 ffff88003f80c288
[  329.919162]  ffff88003ddbeee0 0000000000000000 ffff88003c4e5c48 ffffffff812c8433
[  329.919164] Call Trace:
[  329.919166]  [<ffffffff8125a780>] exofs_fill_super+0x440/0x6e0
[  329.919171]  [<ffffffff812c8433>] ? idr_pre_get+0x53/0x90
[  329.919173]  [<ffffffff8125a340>] ? exofs_read_lookup_dev_table+0x480/0x480
[  329.919176]  [<ffffffff81180e18>] mount_nodev+0x58/0xb0
[  329.919178]  [<ffffffff812594ff>] exofs_mount+0x4f/0x80
[  329.919179]  [<ffffffff81181e13>] mount_fs+0x43/0x1b0
[  329.919182]  [<ffffffff8119b56a>] vfs_kern_mount+0x6a/0xf0
[  329.919183]  [<ffffffff8119bd44>] do_kern_mount+0x54/0x110
[  329.919185]  [<ffffffff8119d4b4>] do_mount+0x1a4/0x830
[  329.919188]  [<ffffffff8115da33>] ? alloc_pages_current+0xa3/0x110
[  329.919191]  [<ffffffff811229e4>] ? __get_free_pages+0x14/0x50
[  329.919193]  [<ffffffff8119d18a>] ? copy_mount_options+0x3a/0x170
[  329.919194]  [<ffffffff8119dc80>] sys_mount+0x90/0xe0
[  329.919199]  [<ffffffff81605aa9>] system_call_fastpath+0x16/0x1b
[  329.919200] Code: 66 90 41 8b 84 24 70 03 00 00 83 eb 01 39 c3 72 50 41 8b 94 24 74 03 00 00 01 c2 39 d3 73 42 89 da 29 c2 49 8b 84 24 88 03 00 00 <48> 8b 04 d0 48 8b 38 48 85 ff 74 0c 48 c7 00 00 00 00 00 e8 6f
[  329.919214] RIP  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919216]  RSP <ffff88003c4e5bc8>
[  329.919217] CR2: 0000000000000000
[  329.919220] ---[ end trace e5a3b1124e42d9e3 ]---

Is there any possibility to avoid these failure?

Just now i am using the 3.3.0 kernel from the linux-pnfs repository.

Thanks in adavance

Cheers
Johannes

-------- Original-Nachricht --------

Quoted 67 lines

Datum: Tue, 15 May 2012 12:48:08 +0300
Von: Boaz Harrosh <[email protected]>
An: Johannes Schild <[email protected]>
CC: [email protected], open-osd <[email protected]>
Betreff: Re: Questions about Exofs

Quoted 5 lines · Johannes Schild

Hi,

i try to set up the object-based layout with pnfs.
I am using the OSC-OSD as target. It works well and i can login. Trying the scripts "do-*" from git://git.open-osd.org/open-osd.git the

do-osd script works for me. The do-exofs doesnt, here i have some question:

Quoted 3 lines

1. Is the UUID in the do-exofs script optional?

No it is not optional

Quoted 2 lines

2. If no how can i get it for my device?

Exactly! the ./do-exofs format command will set this for you, as well as mkfs.exofs the
FS for you.

Quoted 2 lines

3. It is mandatory to use raid-driver (raid456) in the script? Why?

What? where? Rrrr you are right. this is a very old tree. Let me see if I have something newer to push. It should all load automatically now.

But yes the dependency on raid456.ko is built in. Though if you use raid=0 on the format command line it will not be used in run time.

Quoted 3 lines

The error is:
mount -t exofs -o

osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000,_netdev /dev/osd0 /mnt/osd0/

Quoted 8 lines

mount: wrong fs type, bad option, bad superblock on /dev/osd0,
missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so

dmesg says:
exofs: Unable to mount exofs on (null) pid=0x0 err=-22

Probably you forgot the "./do-exofs format" stage. Please read the script before, you might need to edit it. It is the stage that calls mkfs.exofs to make a new filesystem for you. (An OSD is a raw device that can host many filesystems each in it's own osd-partition.

Quoted 7 lines

Thanks for answering my questions.

Regards
Johannes

Quoted 7 lines

Cheers
Boaz

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 7 lines

--
NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone!
Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi,
This crash has happened to me as well. I was not able to explicitly resolve it, but I have managed to set up a working pNFS object layout environment. I'll try to give the instructions I use for setting up the MDS, without using the do-* scripts, assuming you managed to set up the data server. (I'm assuming a fedora distro):

setup the iscsi initiator:

# yum -y install iscsi-initiator-utils
# service iscsid start
# modprobe osd
# iscsiadm -m discovery -t sendtargets -p <your DS> --login

on my setup, in some (most) cases, the MDS machine hangs when you power it off. I don't know why this happens.

format and mount the OSD device:

# modprobe raid456

# modprobe exofs

# cd /your/open-osd/project/directory
# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --dev=/dev/osd0

to me, this works without UUID. I had trouble in setting up exofs with RAID, but if you're not interested in RAID that will do. note that this setup is from the old tree (commit df9cc23ca0e15f2fc7f1d16a21acb87c63038f50), I don't know if and what have been pushed to it recently.

note that if you use the --format flag you would have to supply a UUID.

now you can mount the object file system. /dev/osd0 is seen as a char device and not a block device, so the system can't read its superblock and tell which file system is it. you have to explicitly specify exofs as the file system, and exofs will do the hard work.:

# mount -t exofs -o pid=0x10000 /dev/osd0 /mnt/pnfs

Quoted 161 lines · Idan Kedar

Quoted 135 lines · Johannes Schild

Hi,

thanks for your fast reply.

But now i run into a nullpointer exception while mounting /dev/osd0

[  329.919112] Modules linked in: raid456 async_raid6_recov async_pq raid6_pq async_memcpy bnx2i cnic cxgb4i cxgb3i iscsi_tcp be2iscsi iscsi_boot_sysfs uio cxgb4 libcxgbi cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi fuse ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack 8021q ip6table_filter ip6_tables garp stp llc fcoe libfcoe libfc scsi_transport_fc scsi_tgt snd_ens1371 gameport snd_rawmidi snd_ac97_codec ac97_bus joydev snd_seq snd_seq_device snd_pcm ppdev vmw_balloon floppy parport_pc snd_timer snd pcnet32 microcode parport pcspkr mii snd_page_alloc i2c_piix4 i2c_core shpchp uinput mptspi mptscsih mptbase scsi_transport_spi [last unloaded: iscsi_tcp] [  329.919133]
[  329.919134] Pid: 2234, comm: mount Not tainted 3.3.0-pnfs #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
[  329.919137] RIP: 0010:[<ffffffff81259799>]  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919139] RSP: 0018:ffff88003c4e5bc8  EFLAGS: 00010246
[  329.919141] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000001795
[  329.919142] RDX: 0000000000000000 RSI: 0000000000000046 RDI: ffff88003c4f5000
[  329.919143] RBP: ffff88003c4e5bd8 R08: 0000000000000000 R09: 0000000000000000
[  329.919144] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88003c4f5000
[  329.919145] R13: ffff88003c4e5d58 R14: 00000000fffffffb R15: ffff88003bd4e360
[  329.919146] FS:  00007f68ce81d800(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
[  329.919148] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  329.919149] CR2: 0000000000000000 CR3: 000000003cbfd000 CR4: 00000000000406f0
[  329.919152] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  329.919155] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  329.919156] Process mount (pid: 2234, threadinfo ffff88003c4e4000, task ffff88003aa42e40)
[  329.919157] Stack:
[  329.919158]  ffff88003c4f5000 ffff88003c4f5800 ffff88003c4e5d18 ffffffff8125a780
[  329.919160]  ffff88003c4e5c84 0000000081f27618 ffffffff81f27600 ffff88003f80c288
[  329.919162]  ffff88003ddbeee0 0000000000000000 ffff88003c4e5c48 ffffffff812c8433
[  329.919164] Call Trace:
[  329.919166]  [<ffffffff8125a780>] exofs_fill_super+0x440/0x6e0
[  329.919171]  [<ffffffff812c8433>] ? idr_pre_get+0x53/0x90
[  329.919173]  [<ffffffff8125a340>] ? exofs_read_lookup_dev_table+0x480/0x480
[  329.919176]  [<ffffffff81180e18>] mount_nodev+0x58/0xb0
[  329.919178]  [<ffffffff812594ff>] exofs_mount+0x4f/0x80
[  329.919179]  [<ffffffff81181e13>] mount_fs+0x43/0x1b0
[  329.919182]  [<ffffffff8119b56a>] vfs_kern_mount+0x6a/0xf0
[  329.919183]  [<ffffffff8119bd44>] do_kern_mount+0x54/0x110
[  329.919185]  [<ffffffff8119d4b4>] do_mount+0x1a4/0x830
[  329.919188]  [<ffffffff8115da33>] ? alloc_pages_current+0xa3/0x110
[  329.919191]  [<ffffffff811229e4>] ? __get_free_pages+0x14/0x50
[  329.919193]  [<ffffffff8119d18a>] ? copy_mount_options+0x3a/0x170
[  329.919194]  [<ffffffff8119dc80>] sys_mount+0x90/0xe0
[  329.919199]  [<ffffffff81605aa9>] system_call_fastpath+0x16/0x1b
[  329.919200] Code: 66 90 41 8b 84 24 70 03 00 00 83 eb 01 39 c3 72 50 41 8b 94 24 74 03 00 00 01 c2 39 d3 73 42 89 da 29 c2 49 8b 84 24 88 03 00 00 <48> 8b 04 d0 48 8b 38 48 85 ff 74 0c 48 c7 00 00 00 00 00 e8 6f
[  329.919214] RIP  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919216]  RSP <ffff88003c4e5bc8>
[  329.919217] CR2: 0000000000000000
[  329.919220] ---[ end trace e5a3b1124e42d9e3 ]---

Is there any possibility to avoid these failure?

Just now i am using the 3.3.0 kernel from the linux-pnfs repository.

Thanks in adavance

Cheers
Johannes

-------- Original-Nachricht --------

Quoted 67 lines

Datum: Tue, 15 May 2012 12:48:08 +0300
Von: Boaz Harrosh <[email protected]>
An: Johannes Schild <[email protected]>
CC: [email protected], open-osd <[email protected]>
Betreff: Re: Questions about Exofs

Quoted 5 lines · Johannes Schild

Hi,

i try to set up the object-based layout with pnfs.
I am using the OSC-OSD as target. It works well and i can login. Trying the scripts "do-*" from git://git.open-osd.org/open-osd.git the

do-osd script works for me. The do-exofs doesnt, here i have some question:

Quoted 3 lines

1. Is the UUID in the do-exofs script optional?

No it is not optional

Quoted 2 lines

2. If no how can i get it for my device?

Exactly! the ./do-exofs format command will set this for you, as well as mkfs.exofs the
FS for you.

Quoted 2 lines

3. It is mandatory to use raid-driver (raid456) in the script? Why?

What? where? Rrrr you are right. this is a very old tree. Let me see if I have something newer to push. It should all load automatically now.

But yes the dependency on raid456.ko is built in. Though if you use raid=0 on the format command line it will not be used in run time.

Quoted 3 lines

The error is:
mount -t exofs -o

osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000,_netdev /dev/osd0 /mnt/osd0/

Quoted 8 lines

mount: wrong fs type, bad option, bad superblock on /dev/osd0, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail  or so

dmesg says:
exofs: Unable to mount exofs on (null) pid=0x0 err=-22

Probably you forgot the "./do-exofs format" stage. Please read the script before, you might need to edit it. It is the stage that calls mkfs.exofs to make a new filesystem for you. (An OSD is a raw device that can host many filesystems each in it's own osd-partition.

Quoted 7 lines

Thanks for answering my questions.

Regards
Johannes

Quoted 7 lines

Cheers
Boaz

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Quoted 7 lines

--
NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone!
Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi,
This crash has happened to me as well. I was not able to explicitly resolve it, but I have managed to set up a working pNFS object layout environment. I'll try to give the instructions I use for setting up the MDS, without using the do-* scripts, assuming you managed to set up the data server. (I'm assuming a fedora distro):

setup the iscsi initiator:

# yum -y install iscsi-initiator-utils
# service iscsid start
# modprobe osd
# iscsiadm -m discovery -t sendtargets -p <your DS> --login

on my setup, in some (most) cases, the MDS machine hangs when you power it off. I don't know why this happens.

Quoted 3 lines

format and mount the OSD device:

# modprobe raid456

This should not be needed exofs is dependent on ore, dependent on raid456

Quoted 6 lines

# modprobe exofs

# cd /your/open-osd/project/directory
# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --dev=/dev/osd0

NO you must --format with a UUID at least once.

After the iscsiadm --login stage, there is an osd.ko print in dmesg Do you see a print of the form:
       OSD_NAME               [%s]
where %s might be empty or it might have something
If there is something (not []) then maybe it was specified on the osd-target (otgtd) command line and all is well. But if not then it can never work.

Quoted 8 lines

to me, this works without UUID. I had trouble in setting up exofs with RAID, but if you're not interested in RAID that will do. note that this setup is from the old tree (commit df9cc23ca0e15f2fc7f1d16a21acb87c63038f50), I don't know if and what have been pushed to it recently.

note that if you use the --format flag you would have to supply a UUID.

You must do that at least once in the lifetime of the device or set an osd_name at the otgtd command line.

Quoted 4 lines

now you can mount the object file system. /dev/osd0 is seen as a char device and not a block device, so the system can't read its superblock and tell which file system is it. you have to explicitly specify exofs as the file system, and exofs will do the hard work.:

Whaoo I've been using ./do-exofs for ever and did not realize this. I should fix that. Thanks It's now on my todo list.

Quoted 4 lines

# mount -t exofs -o pid=0x10000 /dev/osd0 /mnt/pnfs

Are you sure you have an exofs FS at /mnt/pnfs ? please do an

# df -h /mnt/pnfs I want to see ?

Does it actually work? OK you know maybe it does. I can see this now. If you have a single device it might work, I didn't realize this. I thought the mkfs.exofs would not let you.

For sure if you have more then one device (pnfs right?) then it will not let you, because the devices have strict order in the device table. Device names are not reliable and may change from login to login. You need some kind of device-id

Quoted 2 lines

Sorry guys for the lack of DOCs and proper trees. I neglected that for a while now. No one cared.

Let me push some better tested trees.

Thanks
Boaz

On Tue, May 15, 2012 at 4:42 PM, Boaz Harrosh <[email protected]> wrote:

Quoted 1 line

This should not be needed exofs is dependent on ore, dependent on raid456

True, but when setting up the environment my goal was to have a working environment. If a bug is introduced to this dependence system and the module wouldn't be loaded, I would have to reluctantly spend time finding the problem, when I can actually live with such a bug if the workaround is simply loading the module manually. so as a policy, I always explicitly load the modules I need when setting up a pNFS environment.

Quoted 3 lines

Are you sure you have an exofs FS at /mnt/pnfs ? please do an

# df -h /mnt/pnfs I want to see ?

# df -hT /mnt/pnfs/
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/osd0    exofs     55G   12G   44G  21% /mnt/pnfs

Quoted 9 lines

Does it actually work? OK you know maybe it does. I can see this now. If you have a single device it might work, I didn't realize this. I thought the mkfs.exofs would not let you.

For sure if you have more then one device (pnfs right?) then it will not let you, because the devices have strict order in the device table. Device names are not reliable and may change from login to login. You need some kind of device-id

Indeed it is a pNFS setup. And indeed I've had trouble when using more than one OSD.

By the way, several weeks I have tried setting up a RAID 5 environment with 8 OSDs, 1 mirror and RAID nesting. I then tried cloning and compiling the kernel tree over this pNFS-OSD-RAID. The result was that otgtd died, and I don't know why. It didn't dump core anywhere I could find and the only "log" it has - stdout - didn't give any useful info. I was going to inquire about this in a couple of weeks when I need to get this environment working, but since this issue came up, maybe we can somehow resolve it sooner.

Quoted 1 line · Idan Kedar

On Tue, May 15, 2012 at 4:42 PM, Boaz Harrosh <[email protected]> wrote:

<snip>

Quoted 10 lines

Quoted 1 line

This should not be needed exofs is dependent on ore, dependent on raid456

True, but when setting up the environment my goal was to have a working environment. If a bug is introduced to this dependence system and the module wouldn't be loaded, I would have to reluctantly spend time finding the problem, when I can actually live with such a bug if the workaround is simply loading the module manually. so as a policy, I always explicitly load the modules I need when setting up a pNFS environment.

I'm sorry about that, that was my mess. My intention was to make all these dependencies totally transparent. As an implementation de-jur detail, not user visible.

The reason I had it in script for a while was because in UML raid456.ko had a bug, which probing it manually gave a 20% better chance of success (Tree below has a patch that fixes that, on UML)

Please note that some versions of exofs need it and some don't. I have removed it from my scripts, now.

<snip>

Quoted 10 lines

Quoted 3 lines

Are you sure you have an exofs FS at /mnt/pnfs ? please do an

# df -h /mnt/pnfs I want to see ?

# df -hT /mnt/pnfs/
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/osd0    exofs     55G   12G   44G  21% /mnt/pnfs

OK I understand that now, the weird single device case. I don't promise it will continue to work in Future. As a rule all devices most have a network-unique OSD_NAME.

Quoted 7 lines

Quoted 7 lines

Does it actually work? OK you know maybe it does. I can see this now. If you have a single device it might work, I didn't realize this. I thought the mkfs.exofs would not let you.

Quoted 6 lines

Quoted 4 lines

For sure if you have more then one device (pnfs right?) then it will not let you, because the devices have strict order in the device table. Device names are not reliable and may change from login to login. You need some kind of device-id

Indeed it is a pNFS setup. And indeed I've had trouble when using more than one OSD.

I use a better script system for the cluster case, now, that I never pushed that makes all this easier.

For one not setting osdname= at --format would explain the above.

Quoted 9 lines

By the way, several weeks I have tried setting up a RAID 5 environment with 8 OSDs, 1 mirror and RAID nesting. I then tried cloning and compiling the kernel tree over this pNFS-OSD-RAID. The result was that otgtd died, and I don't know why. It didn't dump core anywhere I could find and the only "log" it has - stdout - didn't give any useful info. I was going to inquire about this in a couple of weeks when I need to get this environment working, but since this issue came up, maybe we can somehow resolve it sooner.

8 OSDs with a mirror ? what was the mkfs.exofs command line you used?

And did you use one otgtd with 8 targets, or 8 targets (8 IP addresses) with one target each, or a combination?

What is the otgtd platform? what file system? what HW and HD environment?

And yes otgtd has some instabilities.

There are two I can think off:
* Over xfs the --format command crashes the otgtd (aborted exit no crash dump) Debugging welcome.

* When lots of pnfs clients do heavy writing to the same otgtd, it times-out and disconnects.

 At Panasas we have a watch-dog that reloads it in a loop. I have only seen this on FreeBSD, in Linux it never happened to me.

Please give me more details on what you did before it exited like that.

In anyway I pushed a tree I tested with at:
       git://git.open-osd.org/linux-open-osd.git

checkout the *merge_and_compile-3.3* branch. But in principal they are the same:
       fs/exofs                - Added autologin support
       fs/nfs/objlayout        - Added autologin support
       fs/nfsd                 - Same
       fs/nfs                  - Few fixes that are in benny's tree are not in linux-open-osd

So it should all be the same. For a proper cluster setup you will probably need my do-ect scripts which take a cluster descriptor file and does generic loops on everything.

Thanks
Boaz

Quoted 1 line

8 OSDs with a mirror ? what was the mkfs.exofs command line you used?

Something along the lines of
# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --format --mirrors=1 --group_width=2 --group_depth=2 --dev=/dev/osd0 --osdname=$(uuid) --dev=/dev/osd1 --osdname=$(uuid) --dev=/dev/osd2 --osdname=$(uuid) ...

I don't remember exactly at the moment, but I will bump this thread when I'll start using RAID again.

Quoted 3 lines

And did you use one otgtd with 8 targets, or 8 targets (8 IP addresses) with one target each, or a combination?

one target with 8 LUNs

Quoted 2 lines

What is the otgtd platform? what file system? what HW and HD environment?

osc-osd over ext4, 64 bit VirtualBox VM over x86_64.

Quoted 9 lines

And yes otgtd has some instabilities.

There are two I can think off:
* Over xfs the --format command crashes the otgtd (aborted exit no crash dump) Debugging welcome.

* When lots of pnfs clients do heavy writing to the same otgtd, it times-out and disconnects.

it was a single client performing git-clone of the kernel tree.

Quoted 6 lines

At Panasas we have a watch-dog that reloads it in a loop. I have only seen this on FreeBSD, in Linux it never happened to me.

Please give me more details on what you did before it exited like that.

Nothing special, just git-clone. at some point it hanged (at a different place every time), and when investigated a bit I saw that otgtd is dead.

Quoted 11 lines

In anyway I pushed a tree I tested with at:
git://git.open-osd.org/linux-open-osd.git

checkout the *merge_and_compile-3.3* branch. But in principal they are the same:

       fs/exofs                - Added autologin support
       fs/nfs/objlayout        - Added autologin support
       fs/nfsd                 - Same
       fs/nfs                  - Few fixes that are in benny's tree are not in linux-open-osd

Thanks, I will try it soon.

Quoted 4 lines

So it should all be the same. For a proper cluster setup you will probably need my do-ect scripts which take a cluster descriptor file and does generic loops on everything.

Please note that I didn't try a cluster setup, just a single DS with 8 LUNs, single MDS, and single pNFS client, all 3 different VMs on the same host.

Quoted 3 lines

Thanks
Boaz

On 05/15/2012 07:21 PM, Idan Kedar wrote:
<snip>

Quoted 9 lines

Quoted 1 line

8 OSDs with a mirror ? what was the mkfs.exofs command line you used?

Something along the lines of
# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --format --mirrors=1 --group_width=2 --group_depth=2 --dev=/dev/osd0 --osdname=$(uuid) --dev=/dev/osd1 --osdname=$(uuid) --dev=/dev/osd2 --osdname=$(uuid) ...

I don't remember exactly at the moment, but I will bump this thread when I'll start using RAID again.

Certainly missing the --format thing.  --osdname= without an --format is ignored.

Again if you never set an OSD_NAME on the devices in the past this will not work.

But perhaps you did --format and forgot

Quoted 7 lines

Quoted 3 lines

And did you use one otgtd with 8 targets, or 8 targets (8 IP addresses) with one target each, or a combination?

one target with 8 LUNs

Quoted 2 lines

What is the otgtd platform? what file system? what HW and HD environment?

osc-osd over ext4, 64 bit VirtualBox VM over x86_64.

OK that's a fishy setup.

The otgtd is sensitive to timeouts which I never investigated properly. It looks like the OSD_VM-to-host, probably a single link, is very slow and imagine 8 initiators actually banging on the same single slow link. One of the commands times out, probably just a guess

The best for a VM setup, that I have is:

VM1     - exofs+MDS
VM2     - pnfs client.
  (On my dev machine I have VM2+VM1 combined, and mount on localhost)

HOST    - Run otgtd naively on host for best results.
         Best is to spread all targets on as many physical HD devices. Note that multiple targets from the same OSD-host only makes "preformance" sense if they each serve a different spindle. Unless in a simulated test environment as yourself

Which reminds me that upstream tgtd as a few fixes in this area I should integrate.

Quoted 30 lines

Quoted 9 lines

And yes otgtd has some instabilities.

There are two I can think off:
* Over xfs the --format command crashes the otgtd (aborted exit no crash dump) Debugging welcome.

* When lots of pnfs clients do heavy writing to the same otgtd, it times-out and disconnects.

it was a single client performing git-clone of the kernel tree.

Quoted 6 lines

 At Panasas we have a watch-dog that reloads it in a loop. I have only seen this on FreeBSD, in Linux it never happened to me.

Please give me more details on what you did before it exited like that.

Nothing special, just git-clone. at some point it hanged (at a different place every time), and when investigated a bit I saw that otgtd is dead.

Quoted 11 lines

In anyway I pushed a tree I tested with at:
       git://git.open-osd.org/linux-open-osd.git

checkout the *merge_and_compile-3.3* branch. But in principal they are the same:
       fs/exofs                - Added autologin support
       fs/nfs/objlayout        - Added autologin support
       fs/nfsd                 - Same
       fs/nfs                  - Few fixes that are in benny's tree are not in linux-open-osd

Quoted 8 lines

Thanks, I will try it soon.

Quoted 4 lines

So it should all be the same. For a proper cluster setup you will probably need my do-ect scripts which take a cluster descriptor file and does generic loops on everything.

Please note that I didn't try a cluster setup, just a single DS with 8 LUNs, single MDS, and single pNFS client, all 3 different VMs on the same host.

Just semantics so we speak the same language. Yes you do have a cluster.

In pnfs-objects world there are no such thing as DSs there is MDS and there are OSDs. (objects). The OSDs are the equivalent of DSs in "files" and LUNs in "blocks"

A none cluster is when you have a single OSD. (No striping, no multiple devices, what I call the trivial layout)

Quoted 4 lines

Quoted 3 lines

Thanks
Boaz

If you want there is a new tree at:
       git://git.open-osd.org/tests.git

There is one script that does everything. ./do-ect (exofs cluster test)

You edit the ect.conf file (or run ./do-ect -f alternate.conf file)

In turn inside ect.conf you edit your topology and setup. It also points to a device-table file (osds_list=XXX.olst) see lots of *.olst example files.

list of operations.
* Read the scripts and study what they do. They are just a convenience
 not a black-box application.
* Edit a new XXX.olst file
* Edit ect.conf with your setup.
[On MDS]
* ./do-ect login2       - login to all devices specified by osds_list=
* ./do-ect format2      - Set up an FS as specified by ect.conf
* ./do-ect mount2       - mount the exofs file system
* ./do-ect seturi       - If you have the autologin version and want an autologin support.

[On pnfs client]
* ./do-ect login2       - Only if autologin is not enabled. You will need the /sbin/osd_login script, which is part of the newest nfs-utils (Tell me if you can't find it)

* ./do-ect pnfs_start  - mount the pnfs server on pnfs_dir as specified in ect.conf

And there are other facilities as well. In principal the commands that end with "2" are those that preform an action on the XXX.olst file and receive an optional parameter as the OSDs list. For example:

./do-ect login2
       - login to default list specified in ect.conf

./do-ect -f clusterXYZ.conf format2 device_table7.olst
       - Format according to setup in clusterXYZ.conf file but override the OSDs list instead use device_table7.olst.

Again please read the scripts before using. the .conf and .olst files are yours, I just have them in git as an history of the tests I conducted.

But if you have any changes to the do-ect and fn-osd.sh please send me a patch.

There are other interesting scripts in there for example the target/ dir as a
way of controlling lots of OSD hosts in a single command using the closh script (CLuster Output SHell) which also operate on the .olst and .clst files. Have fun

Cheers
Boaz

Quoted 11 lines

Quoted 11 lines

By the way, several weeks I have tried setting up a RAID 5 environment

Quoted 7 lines

Quoted 7 lines

with 8 OSDs, 1 mirror and RAID nesting. I then tried cloning and compiling the kernel tree over this pNFS-OSD-RAID. The result was that otgtd died, and I don't know why. It didn't dump core anywhere I could find and the only "log" it has - stdout - didn't give any useful info. I was going to inquire about this in a couple of weeks when I need to get this environment working, but since this issue came up, maybe we can somehow resolve it sooner.

8 OSDs with a mirror ? what was the mkfs.exofs command line you used?

Something along the lines of
# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --format --mirrors=1 --group_width=2 --group_depth=2 --dev=/dev/osd0 --osdname=$(uuid) --dev=/dev/osd1 --osdname=$(uuid) --dev=/dev/osd2 --osdname=$(uuid) ...

I don't remember exactly at the moment, but I will bump this thread when I'll start using RAID again.

Quoted 1 line

Quoted 1 line

Quoted 11 lines · Idan Kedar

Quoted 11 lines

Quoted 11 lines

By the way, several weeks I have tried setting up a RAID 5 environment

Quoted 7 lines

Quoted 7 lines

with 8 OSDs, 1 mirror and RAID nesting. I then tried cloning and compiling the kernel tree over this pNFS-OSD-RAID. The result was that otgtd died, and I don't know why. It didn't dump core anywhere I could find and the only "log" it has - stdout - didn't give any useful info. I was going to inquire about this in a couple of weeks when I need to get this environment working, but since this issue came up, maybe we can somehow resolve it sooner.

8 OSDs with a mirror ? what was the mkfs.exofs command line you used?

Quoted 9 lines

Something along the lines of
# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000 --format --mirrors=1 --group_width=2 --group_depth=2 --dev=/dev/osd0 --osdname=$(uuid) --dev=/dev/osd1 --osdname=$(uuid) --dev=/dev/osd2 --osdname=$(uuid) ...

I don't remember exactly at the moment, but I will bump this thread when I'll start using RAID again.

Quoted 1 line

Quoted 1 line

BTW I don't see a --raid=5 above but a raid5 of --group_width=2 is just a mirror. The minimum for --raid=5 is 3. (I'm not sure if the system checks and optimizes for this)

So the above should probably be:

# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000    \
 --mirrors=2 --group_width=2 --group_depth=1000        \
 --dev=/dev/osd0 --format --osdname=$(uuid)            \
 --dev=/dev/osd1 --format --osdname=$(uuid)            \
 --dev=/dev/osd2 --format --osdname=$(uuid)            \
 ...

group_depth is how many times to repeat this group until advancing to the next group.

The group_count is:
       group_count = int( (total_devices / mirrors+1) / group_width )

an interesting raid5 configuration with 8 devices mirrors and groups can be:

# LD_LIBRARY_PATH=lib ./usr/mkfs.exofs --pid=0x10000    \
 --mirrors=1 --raid=5 --group_width=3 --group_depth=1000       \
 --dev=/dev/osd0 --format --osdname=$(uuid)            \
 --dev=/dev/osd1 --format --osdname=$(uuid)            \
 --dev=/dev/osd2 --format --osdname=$(uuid)            \

Which means each file will have only a single group  - 3-out-of-4. but the next file will use another set of 3 devices out of the 4.

And I think you need a --stripe_pages= right?

Cheers
Boaz

Hi,

thanks for your fast reply.

But now i run into a nullpointer exception while mounting /dev/osd0

[ 329.919112] Modules linked in: raid456 async_raid6_recov async_pq
		raid6_pq async_memcpy bnx2i cnic cxgb4i cxgb3i iscsi_tcp be2iscsi
		iscsi_boot_sysfs uio cxgb4 libcxgbi cxgb3 mdio libiscsi_tcp libiscsi
		scsi_transport_iscsi fuse ip6t_REJECT nf_conntrack_ipv6
		nf_defrag_ipv6 xt_state nf_conntrack 8021q ip6table_filter ip6_tables
		garp stp llc fcoe libfcoe libfc scsi_transport_fc scsi_tgt
		snd_ens1371 gameport snd_rawmidi snd_ac97_codec ac97_bus joydev
		snd_seq snd_seq_device snd_pcm ppdev vmw_balloon floppy parport_pc
		snd_timer snd pcnet32 microcode parport pcspkr mii snd_page_alloc
		i2c_piix4 i2c_core shpchp uinput mptspi mptscsih mptbase
		scsi_transport_spi [last unloaded: iscsi_tcp] [ 329.919133]

[  329.919134] Pid: 2234, comm: mount Not tainted 3.3.0-pnfs #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
[  329.919137] RIP: 0010:[<ffffffff81259799>]  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919139] RSP: 0018:ffff88003c4e5bc8  EFLAGS: 00010246
[  329.919141] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000001795
[  329.919142] RDX: 0000000000000000 RSI: 0000000000000046 RDI: ffff88003c4f5000
[  329.919143] RBP: ffff88003c4e5bd8 R08: 0000000000000000 R09: 0000000000000000
[  329.919144] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88003c4f5000
[  329.919145] R13: ffff88003c4e5d58 R14: 00000000fffffffb R15: ffff88003bd4e360
[  329.919146] FS:  00007f68ce81d800(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
[  329.919148] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  329.919149] CR2: 0000000000000000 CR3: 000000003cbfd000 CR4: 00000000000406f0
[  329.919152] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  329.919155] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  329.919156] Process mount (pid: 2234, threadinfo ffff88003c4e4000, task ffff88003aa42e40)
[  329.919157] Stack:
[  329.919158]  ffff88003c4f5000 ffff88003c4f5800 ffff88003c4e5d18 ffffffff8125a780
[  329.919160]  ffff88003c4e5c84 0000000081f27618 ffffffff81f27600 ffff88003f80c288
[  329.919162]  ffff88003ddbeee0 0000000000000000 ffff88003c4e5c48 ffffffff812c8433
[  329.919164] Call Trace:
[  329.919166]  [<ffffffff8125a780>] exofs_fill_super+0x440/0x6e0
[  329.919171]  [<ffffffff812c8433>] ? idr_pre_get+0x53/0x90
[  329.919173]  [<ffffffff8125a340>] ? exofs_read_lookup_dev_table+0x480/0x480
[  329.919176]  [<ffffffff81180e18>] mount_nodev+0x58/0xb0
[  329.919178]  [<ffffffff812594ff>] exofs_mount+0x4f/0x80
[  329.919179]  [<ffffffff81181e13>] mount_fs+0x43/0x1b0
[  329.919182]  [<ffffffff8119b56a>] vfs_kern_mount+0x6a/0xf0
[  329.919183]  [<ffffffff8119bd44>] do_kern_mount+0x54/0x110
[  329.919185]  [<ffffffff8119d4b4>] do_mount+0x1a4/0x830
[  329.919188]  [<ffffffff8115da33>] ? alloc_pages_current+0xa3/0x110
[  329.919191]  [<ffffffff811229e4>] ? __get_free_pages+0x14/0x50

[  329.919193]  [<ffffffff8119d18a>] ? copy_mount_options+0x3a/0x170
[  329.919194]  [<ffffffff8119dc80>] sys_mount+0x90/0xe0
[  329.919199]  [<ffffffff81605aa9>] system_call_fastpath+0x16/0x1b
[  329.919200] Code: 66 90 41 8b 84 24 70 03 00 00 83 eb 01 39 c3 72 50 41 8b 94 24 74 03 00 00 01 c2 39 d3 73 42 89 da 29 c2 49 8b 84 24 88 03 00 00 <48> 8b 04 d0 48 8b 38 48 85 ff 74 0c 48 c7 00 00 00 00 00 e8 6f 
[  329.919214] RIP  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919216]  RSP <ffff88003c4e5bc8>
[  329.919217] CR2: 0000000000000000
[  329.919220] ---[ end trace e5a3b1124e42d9e3 ]---

list *(exofs_free_sbi+0x59)

list *(exofs_fill_super+0x440)

Is there any possibility to avoid these failure?

Just now i am using the 3.3.0 kernel from the linux-pnfs repository.

Thanks in adavance

Cheers
Johannes

-------- Original-Nachricht --------

Quoted 74 lines

Datum: Tue, 15 May 2012 12:48:08 +0300
Von: Boaz Harrosh <[email protected]>
An: Johannes Schild <[email protected]>
CC: [email protected], open-osd <[email protected]>
Betreff: Re: Questions about Exofs

Quoted 5 lines · Johannes Schild

Hi,

i try to set up the object-based layout with pnfs.
I am using the OSC-OSD as target. It works well and i can login. Trying the scripts "do-*" from git://git.open-osd.org/open-osd.git the

do-osd script works for me. The do-exofs doesnt, here i have some question:

Quoted 3 lines

1. Is the UUID in the do-exofs script optional?

No it is not optional

Quoted 2 lines

2. If no how can i get it for my device?

Exactly! the ./do-exofs format command will set this for you, as well as mkfs.exofs the
FS for you.

Quoted 2 lines

3. It is mandatory to use raid-driver (raid456) in the script? Why?

What? where? Rrrr you are right. this is a very old tree. Let me see if I have something newer to push. It should all load automatically now.

But yes the dependency on raid456.ko is built in. Though if you use raid=0 on the format command line it will not be used in run time.

Quoted 3 lines

The error is:
mount -t exofs -o

osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000,_netdev /dev/osd0 /mnt/osd0/

Quoted 8 lines

mount: wrong fs type, bad option, bad superblock on /dev/osd0,
missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so

dmesg says:
exofs: Unable to mount exofs on (null) pid=0x0 err=-22

Probably you forgot the "./do-exofs format" stage. Please read the script before, you might need to edit it. It is the stage that calls mkfs.exofs to make a new filesystem for you. (An OSD is a raw device that can host many filesystems each in it's own osd-partition.

Quoted 7 lines

Thanks for answering my questions.

Regards
Johannes

Cheers
Boaz

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Datum: Tue, 15 May 2012 16:18:17 +0300
Von: Boaz Harrosh <[email protected]>
An: Johannes Schild <[email protected]>
CC: [email protected], [email protected]
Betreff: Re: Questions about Exofs

Quoted 8 lines · Johannes Schild

Hi,

thanks for your fast reply.

But now i run into a nullpointer exception while mounting /dev/osd0

Quoted 11 lines

[ 329.919112] Modules linked in: raid456 async_raid6_recov async_pq
		raid6_pq async_memcpy bnx2i cnic cxgb4i cxgb3i iscsi_tcp be2iscsi
		iscsi_boot_sysfs uio cxgb4 libcxgbi cxgb3 mdio libiscsi_tcp libiscsi
		scsi_transport_iscsi fuse ip6t_REJECT nf_conntrack_ipv6
		nf_defrag_ipv6 xt_state nf_conntrack 8021q ip6table_filter ip6_tables
		garp stp llc fcoe libfcoe libfc scsi_transport_fc scsi_tgt
		snd_ens1371 gameport snd_rawmidi snd_ac97_codec ac97_bus joydev
		snd_seq snd_seq_device snd_pcm ppdev vmw_balloon floppy parport_pc
		snd_timer snd pcnet32 microcode parport pcspkr mii snd_page_alloc
		i2c_piix4 i2c_core shpchp uinput mptspi mptscsih mptbase
		scsi_transport_spi [last unloaded: iscsi_tcp] [ 329.919133]

One thing that worries me here is this: [last unloaded: iscsi_tcp]

Actually it should be in the loaded bunch not unloaded. I have no idea how can that happen.
Do you see any prints in dmsg regarding iscsi, before the crash?

Quoted 2 lines

[  329.919134] Pid: 2234, comm: mount Not tainted 3.3.0-pnfs #1 VMware,

Inc. VMware Virtual Platform/440BX Desktop Reference Platform

Quoted 1 line

[  329.919137] RIP: 0010:[<ffffffff81259799>]  [<ffffffff81259799>]

exofs_free_sbi+0x59/0xa0
> [  329.919139] RSP: 0018:ffff88003c4e5bc8  EFLAGS: 00010246
> [  329.919141] RAX: 0000000000000000 RBX: 0000000000000000 RCX:

0000000000001795

Quoted 1 line

[  329.919142] RDX: 0000000000000000 RSI: 0000000000000046 RDI:

ffff88003c4f5000

Quoted 1 line

[  329.919143] RBP: ffff88003c4e5bd8 R08: 0000000000000000 R09:

0000000000000000

Quoted 1 line

[  329.919144] R10: 0000000000000000 R11: 0000000000000001 R12:

ffff88003c4f5000

Quoted 1 line

[  329.919145] R13: ffff88003c4e5d58 R14: 00000000fffffffb R15:

ffff88003bd4e360

Quoted 1 line

[  329.919146] FS:  00007f68ce81d800(0000) GS:ffff88003fc00000(0000)

knlGS:0000000000000000

Quoted 2 lines

[  329.919148] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  329.919149] CR2: 0000000000000000 CR3: 000000003cbfd000 CR4:

00000000000406f0

Quoted 1 line

[  329.919152] DR0: 0000000000000000 DR1: 0000000000000000 DR2:

0000000000000000

Quoted 1 line

[  329.919155] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:

0000000000000400

Quoted 1 line

[  329.919156] Process mount (pid: 2234, threadinfo ffff88003c4e4000,

task ffff88003aa42e40)

Quoted 2 lines

[  329.919157] Stack:
[  329.919158]  ffff88003c4f5000 ffff88003c4f5800 ffff88003c4e5d18

ffffffff8125a780

Quoted 1 line

[  329.919160]  ffff88003c4e5c84 0000000081f27618 ffffffff81f27600

ffff88003f80c288

Quoted 1 line

[  329.919162]  ffff88003ddbeee0 0000000000000000 ffff88003c4e5c48

ffffffff812c8433

Quoted 4 lines

[  329.919164] Call Trace:
[  329.919166]  [<ffffffff8125a780>] exofs_fill_super+0x440/0x6e0
[  329.919171]  [<ffffffff812c8433>] ? idr_pre_get+0x53/0x90
[  329.919173]  [<ffffffff8125a340>] ?

exofs_read_lookup_dev_table+0x480/0x480
> [  329.919176]  [<ffffffff81180e18>] mount_nodev+0x58/0xb0
> [  329.919178]  [<ffffffff812594ff>] exofs_mount+0x4f/0x80
> [  329.919179]  [<ffffffff81181e13>] mount_fs+0x43/0x1b0
> [  329.919182]  [<ffffffff8119b56a>] vfs_kern_mount+0x6a/0xf0
> [  329.919183]  [<ffffffff8119bd44>] do_kern_mount+0x54/0x110
> [  329.919185]  [<ffffffff8119d4b4>] do_mount+0x1a4/0x830
> [  329.919188]  [<ffffffff8115da33>] ? alloc_pages_current+0xa3/0x110
> [  329.919191]  [<ffffffff811229e4>] ? __get_free_pages+0x14/0x50
> [  329.919193]  [<ffffffff8119d18a>] ? copy_mount_options+0x3a/0x170
> [  329.919194]  [<ffffffff8119dc80>] sys_mount+0x90/0xe0
> [  329.919199]  [<ffffffff81605aa9>] system_call_fastpath+0x16/0x1b

Quoted 1 line

[  329.919200] Code: 66 90 41 8b 84 24 70 03 00 00 83 eb 01 39 c3 72 50

41 8b 94 24 74 03 00 00 01 c2 39 d3 73 42 89 da 29 c2 49 8b 84 24 88 03 00 00 <48> 8b 04 d0 48 8b 38 48 85 ff 74 0c 48 c7 00 00 00 00 00 e8 6f

Quoted 6 lines

[  329.919214] RIP  [<ffffffff81259799>] exofs_free_sbi+0x59/0xa0
[  329.919216]  RSP <ffff88003c4e5bc8>
[  329.919217] CR2: 0000000000000000
[  329.919220] ---[ end trace e5a3b1124e42d9e3 ]---

could you please do:
[]$ gdb fs/exofs/exofs.ko

Inside gdb

Quoted 1 line

list *(exofs_free_sbi+0x59)

and also

Quoted 1 line

list *(exofs_fill_super+0x440)

Quoted 3 lines

Is there any possibility to avoid these failure?

I've never had it lets try to resolve it.

Could you enable CONFIG_EXOFS_DEBUG it's under:
miscellaneous-filesystems/exofs in make xconfig

Then re-run everything send me the output
[]$ ./do-osd stop

[]$ ls /dev/osd*

[]$ ./do-osd

[]$ ls /dev/osd*

[]$ ./do-exofs format
Send me the output of that

[]$ ./do-exofs start
Send me the dmesg output of this stage, or if not too big the dmesg output of from before ./do-osd <1>

Quoted 3 lines

Just now i am using the 3.3.0 kernel from the linux-pnfs repository.

When compiling the Kernel, Did you enable CONFIG_PNFSD ? (That is the pNFSD Server Kernel Support)

What platform are you using? Distro + ARCH ?

This should be a good tree. Though usually I use an open-osd.org/linux-open-osd.git based tree. But I don't see much difference. Let me have a look if there is something grossly missing.

Quoted 3 lines

Thanks in adavance

Sorry for your trouble I have neglected the public trees for a while. Let me see if I can push a more tested tree up on open-osd.org.

Quoted 3 lines

Cheers
Johannes

Thanks
Boaz

Quoted 15 lines

-------- Original-Nachricht --------

Quoted 14 lines

Datum: Tue, 15 May 2012 12:48:08 +0300
Von: Boaz Harrosh <[email protected]>
An: Johannes Schild <[email protected]>
CC: [email protected], open-osd <[email protected]>
Betreff: Re: Questions about Exofs

Quoted 5 lines · Johannes Schild

Hi,

i try to set up the object-based layout with pnfs.
I am using the OSC-OSD as target. It works well and i can login. Trying the scripts "do-*" from git://git.open-osd.org/open-osd.git the

do-osd script works for me. The do-exofs doesnt, here i have some

question:

Quoted 5 lines

Quoted 5 lines

Quoted 3 lines

1. Is the UUID in the do-exofs script optional?

Quoted 7 lines

Quoted 7 lines

No it is not optional

Quoted 2 lines

2. If no how can i get it for my device?

Exactly! the ./do-exofs format command will set this for you, as well

as

Quoted 9 lines

Quoted 9 lines

mkfs.exofs the
FS for you.

Quoted 2 lines

3. It is mandatory to use raid-driver (raid456) in the script? Why?

What? where? Rrrr you are right. this is a very old tree. Let me see if I have something newer to push. It should all load automatically

now.

Quoted 8 lines

Quoted 8 lines

But yes the dependency on raid456.ko is built in. Though if you use raid=0 on the format command line it will not be used in run time.

Quoted 3 lines

The error is:
mount -t exofs -o

osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000,_netdev

/dev/osd0 /mnt/osd0/

Quoted 10 lines

Quoted 10 lines

Quoted 8 lines

mount: wrong fs type, bad option, bad superblock on /dev/osd0,
missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so

dmesg says:
exofs: Unable to mount exofs on (null) pid=0x0 err=-22

Quoted 1 line

Quoted 1 line

Probably you forgot the "./do-exofs format" stage. Please read the

script

Quoted 1 line

Quoted 1 line

before, you might need to edit it. It is the stage that calls

mkfs.exofs

Quoted 20 lines

Quoted 19 lines

to make a new filesystem for you. (An OSD is a raw device that can host many filesystems each in it's own osd-partition.

Quoted 7 lines

Thanks for answering my questions.

Regards
Johannes

Cheers
Boaz

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Boaz,

Quoted 1 line

Do you see any prints in dmsg regarding iscsi, before the crash?

I see output like this. Always "registered" no unloading execpt after the crash.

[    4.713107] iscsi: registered transport (tcp)
#<some output removed>
[    4.739465] iscsi: registered transport (cxgb3i)
#<some output removed>
[    4.750756] iscsi: registered transport (cxgb4i)
#<some output removed>
[    4.771300] iscsi: registered transport (bnx2i)
[    4.781045] iscsi: registered transport (be2iscsi)

Quoted 2 lines

could you please do:
[]$ gdb fs/exofs/exofs.ko

[root@ExB osd-repo]# gdb /root/pnfs-repo/fs/exofs/exofs.ko
GNU gdb (GDB) Fedora (7.3.50.20110722-13.fc16)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /root/pnfs-repo/fs/exofs/exofs.ko...done.

Quoted 2 lines

Inside gdb

Quoted 1 line

list *(exofs_free_sbi+0x59)

(gdb) list *(exofs_free_sbi+0x59)
0x47a9 is in exofs_free_sbi (include/scsi/osd_ore.h:83).
78 /* ore_comp_dev Recievies a logical device index */
79 static inline struct osd_dev *ore_comp_dev(
80 const struct ore_components *oc, unsigned i)
81 {
82 BUG_ON((i < oc->first_dev) || (oc->first_dev + oc->numdevs <= i));
83 return oc->ods[i - oc->first_dev]->od;
84 }
85
86 static inline void ore_comp_set_dev(
87 struct ore_components *oc, unsigned i, struct osd_dev *od)

Quoted 2 lines

and also

Quoted 1 line

list *(exofs_fill_super+0x440)

(gdb) list *(exofs_fill_super+0x440)
0x5850 is in exofs_fill_super (fs/exofs/super.c:847).
842 dput(sb->s_root);
843 sb->s_root = NULL;
844 goto free_sbi;
845 }
846
847 _exofs_print_device("Mounting", opts->dev_name,
848 ore_comp_dev(&sbi->oc, 0),
849 sbi->one_comp.obj.partition);
850 return 0;
851
(gdb)

Quoted 3 lines

Could you enable CONFIG_EXOFS_DEBUG it's under:
miscellaneous-filesystems/exofs in make xconfig

I enabled it.

Quoted 2 lines

Then re-run everything send me the output
[]$ ./do-osd stop

[root@ExB osd-repo]# ./do-osd stop
/dev/osd0
FATAL: Module osd is builtin

Should it be a modul or doesn't matter?

Quoted 1 line

[]$ ls /dev/osd*

[root@ExB osd-repo]# ls /dev/osd*
ls: cannot access /dev/osd*: No such file or directory

Quoted 1 line

[]$ ./do-osd

[root@ExB osd-repo]# ./do-osd
iscsid.service - LSB: Starts and stops login iSCSI daemon.
Loaded: loaded (/etc/rc.d/init.d/iscsid)
Active: inactive (dead) since Wed, 16 May 2012 10:46:23 +0200; 3min 11s ago
Process: 2287 ExecStop=/etc/rc.d/init.d/iscsid stop (code=exited, status=0/SUCCESS)
Process: 1168 ExecStart=/etc/rc.d/init.d/iscsid start (code=exited, status=0/SUCCESS)
Main PID: 1213 (code=exited, status=0/SUCCESS)
CGroup: name=systemd:/system/iscsid.service
18446744072101122080
login into: 192.168.0.1:3260
192.168.0.1:3260,1 .root.var.osd-tgt.tgt-1.ExA

Quoted 1 line

[]$ ls /dev/osd*

[root@ExB server]# ls /dev/os*
/dev/osd1

Quoted 2 lines

[]$ ./do-exofs format
Send me the output of that

./do-exofs format
mkexofs_format >>>

osd stop? >>>
FATAL: Module osd is builtin
osd start? >>>
iscsid.service - LSB: Starts and stops login iSCSI daemon.
Loaded: loaded (/etc/rc.d/init.d/iscsid)
Active: inactive (dead) since Wed, 16 May 2012 10:46:23 +0200; 6min ago
Process: 2287 ExecStop=/etc/rc.d/init.d/iscsid stop (code=exited, status=0/SUCCESS)
Process: 1168 ExecStart=/etc/rc.d/init.d/iscsid start (code=exited, status=0/SUCCESS)
Main PID: 1213 (code=exited, status=0/SUCCESS)
CGroup: name=systemd:/system/iscsid.service
18446744072101122080
login into: 192.168.0.1:3260
192.168.0.1:3260,1 .root.var.osd-tgt.tgt-1.ExA
Logging in to [iface: default, target: .root.var.osd-tgt.tgt-1.ExA, portal: 192.168.0.1,3260] (multiple) Login to [iface: default, target: .root.var.osd-tgt.tgt-1.ExA, portal: 192.168.0.1,3260] successful.

Quoted 3 lines

[]$ ./do-exofs start
Send me the dmesg output of this stage, or if not too big the dmesg output of from before ./do-osd <1>

I pushed it on nopaste:
http://nopaste.info/cd3c6f9141.html

Quoted 5 lines

Quoted 5 lines

Just now i am using the 3.3.0 kernel from the linux-pnfs repository.

Quoted 4 lines

When compiling the Kernel, Did you enable CONFIG_PNFSD ? (That is the pNFSD Server Kernel Support)

No pNFSD Server support wasn't enabled, i recompiled and activate it

Quoted 4 lines

What platform are you using? Distro + ARCH ?

Iam experimenting with Fedora 16 (3.3.0 pnfs kernel) and arch x86_64

Thanks for your efforts
Johannes

Datum: Wed, 16 May 2012 13:30:34 +0300
Von: Boaz Harrosh <[email protected]>
An: Johannes Schild <[email protected]>
CC: [email protected], [email protected]
Betreff: Re: Questions about Exofs

Quoted 1 line · Johannes Schild

Hi Boaz,

<>

Quoted 3 lines

Quoted 1 line

Do you see any prints in dmsg regarding iscsi, before the crash?

I see output like this. Always "registered" no unloading execpt after

the crash.

Quoted 10 lines

[    4.713107] iscsi: registered transport (tcp)
#<some output removed>
[    4.739465] iscsi: registered transport (cxgb3i)
#<some output removed>
[    4.750756] iscsi: registered transport (cxgb4i)
#<some output removed>
[    4.771300] iscsi: registered transport (bnx2i)
[    4.781045] iscsi: registered transport (be2iscsi)

<>

Quoted 7 lines

Quoted 2 lines

could you please do:
[]$ gdb fs/exofs/exofs.ko

[root@ExB osd-repo]# gdb /root/pnfs-repo/fs/exofs/exofs.ko GNU gdb (GDB) Fedora (7.3.50.20110722-13.fc16) Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later

<http://gnu.org/licenses/gpl.html>

Quoted 2 lines

This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show

copying"

Quoted 25 lines

and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /root/pnfs-repo/fs/exofs/exofs.ko...done.

Quoted 2 lines

Inside gdb

Quoted 1 line

list *(exofs_free_sbi+0x59)

(gdb) list *(exofs_free_sbi+0x59)
0x47a9 is in exofs_free_sbi (include/scsi/osd_ore.h:83).
78 /* ore_comp_dev Recievies a logical device index */
79 static inline struct osd_dev *ore_comp_dev(
80 const struct ore_components *oc, unsigned i)
81 {
82 BUG_ON((i < oc->first_dev) || (oc->first_dev + oc->numdevs <= i));
83 return oc->ods[i - oc->first_dev]->od;
84 }
85
86 static inline void ore_comp_set_dev(
87 struct ore_components *oc, unsigned i, struct osd_dev *od)

Quoted 2 lines

and also

Quoted 1 line

list *(exofs_fill_super+0x440)

Quoted 14 lines

(gdb) list *(exofs_fill_super+0x440)
0x5850 is in exofs_fill_super (fs/exofs/super.c:847).
842 dput(sb->s_root);
843 sb->s_root = NULL;
844 goto free_sbi;
845 }
846
847 _exofs_print_device("Mounting", opts->dev_name,
848 ore_comp_dev(&sbi->oc, 0),
849 sbi->one_comp.obj.partition);
850 return 0;
851
(gdb)

OK I understand we are _exofs_print_device an array that does not exists yet.

Quoted 6 lines

Quoted 3 lines

Could you enable CONFIG_EXOFS_DEBUG it's under:
miscellaneous-filesystems/exofs in make xconfig

I enabled it.

Quoted 9 lines

Quoted 2 lines

Then re-run everything send me the output
[]$ ./do-osd stop

[root@ExB osd-repo]# ./do-osd stop
/dev/osd0
FATAL: Module osd is builtin

Should it be a modul or doesn't matter?

It should be fine. scripts expect it as a module.

Quoted 11 lines

Quoted 1 line

[]$ ls /dev/osd*

[root@ExB osd-repo]# ls /dev/osd*
ls: cannot access /dev/osd*: No such file or directory

Quoted 1 line

[]$ ./do-osd

[root@ExB osd-repo]# ./do-osd
iscsid.service - LSB: Starts and stops login iSCSI daemon.

	  Loaded: loaded (/etc/rc.d/init.d/iscsid)
	  Active: inactive (dead) since Wed, 16 May 2012 10:46:23 +0200; 3min

11s ago

Quoted 1 line

Process: 2287 ExecStop=/etc/rc.d/init.d/iscsid stop (code=exited,

status=0/SUCCESS)

Quoted 1 line

Process: 1168 ExecStart=/etc/rc.d/init.d/iscsid start (code=exited,

status=0/SUCCESS)

Quoted 8 lines

	Main PID: 1213 (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/iscsid.service

18446744072101122080
login into: 192.168.0.1:3260
192.168.0.1:3260,1 .root.var.osd-tgt.tgt-1.ExA

Quoted 1 line

[]$ ls /dev/osd*

Quoted 3 lines

[root@ExB server]# ls /dev/os*
/dev/osd1

/dev/osd1 interesting. make sure your scripts are using /dev/osd1. I suspect this is an artifact of the last games. On a clean reboot a single device should be /dev/osd0. The scripts expect that.

Quoted 5 lines

Quoted 2 lines

[]$ ./do-exofs format
Send me the output of that

./do-exofs format
mkexofs_format >>>

No output from the format command? that is not good. mkfs.exofs is very bad in not saying anything when failing.

Probably because it was formatting /dev/osd0 and we have /dev/osd1 only

Quoted 6 lines

osd stop? >>>
FATAL: Module osd is builtin
osd start? >>>
iscsid.service - LSB: Starts and stops login iSCSI daemon.

	  Loaded: loaded (/etc/rc.d/init.d/iscsid)
	  Active: inactive (dead) since Wed, 16 May 2012 10:46:23 +0200; 6min

ago

Quoted 1 line

Process: 2287 ExecStop=/etc/rc.d/init.d/iscsid stop (code=exited,

status=0/SUCCESS)

Quoted 1 line

Process: 1168 ExecStart=/etc/rc.d/init.d/iscsid start (code=exited,

status=0/SUCCESS)

Quoted 6 lines

	Main PID: 1213 (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/iscsid.service

18446744072101122080
login into: 192.168.0.1:3260
192.168.0.1:3260,1 .root.var.osd-tgt.tgt-1.ExA
Logging in to [iface: default, target: .root.var.osd-tgt.tgt-1.ExA,

portal: 192.168.0.1,3260] (multiple)

Quoted 1 line

Login to [iface: default, target: .root.var.osd-tgt.tgt-1.ExA, portal:

192.168.0.1,3260] successful.

Quoted 8 lines

Quoted 4 lines

[]$ ./do-exofs start
Send me the dmesg output of this stage, or if not too big the dmesg output of from before ./do-osd <1>

I pushed it on nopaste:
http://nopaste.info/cd3c6f9141.html

in the dmesg I see:

[ 2516.994781] exofs @parse_options:88: parse_options
osdname=d2683732-c906-4ee1-9dbd-c10c27bb40df,pid=0x10000
[ 2516.994808] osd @_mach_odi:261: found device sysid_len=0 osdname=36
[ 2516.994816] osd @_osdv2_req_encode_common:617: OSDv2 execute opcode
0x8885
[ 2516.994831] osd @_init_blk_request:1616: or=ffff880020d7ec00 has_in=1

has_out=0 => 0, ffff88003bbf8a10

the very first read below fails. This is the first read from super-block object.
Here it gets an -5 (-EIO) if it was an osd-target error you would have a scsi-sense printout so it means it is a communication problem.

[ 2516.996034] exofs @exofs_read_kern:245: osd_execute_request() => -5
[ 2516.996041] exofs: Unable to mount exofs on (null) pid=0x10000 err=-5

This crash below I should fix. Code is not dealing properly with the IO error
and continues to try and dmesg-print an array that does not exist yet. I will fix that.

[ 2516.996106] BUG: unable to handle kernel NULL pointer dereference at   
      (null)
[ 2516.996111] IP: [<ffffffffa033c779>] exofs_free_sbi+0x59/0xa0 [exofs] 

But the problem still remains why do we get IO errors from iscsi?

Later we have:

[ 3241.802074]  connection1:0: detected conn error (1020)

disconnect. Do you see some prints at the otgtd side. If you use the ./up script it might rederect these to a log file do "./up log"

[ 3398.831629] Chelsio T3 iSCSI Driver cxgb3i v2.0.0 (Jun. 2010)
[ 3398.831919] iscsi: registered transport (cxgb3i)
[ 3398.836776] Chelsio T4 iSCSI Driver cxgb4i v0.9.1 (Aug. 2010)
[ 3398.836996] iscsi: registered transport (cxgb4i)
[ 3398.841397] cnic: Broadcom NetXtreme II CNIC Driver cnic v2.5.8 (Jan 3,
2012)
[ 3398.845267] Broadcom NetXtreme II iSCSI Driver bnx2i v2.7.0.3 (Jun 15,
2011)
[ 3398.845475] iscsi: registered transport (bnx2i)
[ 3400.201828] scsi4 : iSCSI Initiator over TCP/IP
[ 3400.715101] scsi 4:0:0:0: Object storage    IET      OSD             
0001 PQ: 0 ANSI: 5
[ 3400.718038] osd @__detect_osd:359: start scsi_test_unit_ready

ffff880020db3800 ffff880020dfa000 ffff88003974aca0

Right after the crash. So iscsi unloaded and loaded. There was a disconnect.
We must investigate why iscsi has communication problems?

the "192.168.0.1:3260" above is that your host's IP? You are running the otgtd on
the host and exofs in VM? That's good that's what I use all the time.

If you have time you should do two experiments.

1. Please run the "./do-osd test" test. send me the output. It runs a user mode test of the osd device and does some very basic communications.
Note that it will wipe your OSD and you will need to ./do-exofs format
again
after it.

2. on the osd-target side you probably ran ./up. the otgtd also supports none-osd regular disk-devices. Could you set up a regular disk backbend as well. Look into "man tgtadm" on how to add a second disk target.

Once you login to the target you will see a new /dev/sdX device try to dd into it, and also mkfs and mount an ext FS on it.

Or else investigate why there are iscsi communication problems.

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

Just now i am using the 3.3.0 kernel from the linux-pnfs repository.

That's perfect it should have everything.

Quoted 7 lines

Quoted 4 lines

When compiling the Kernel, Did you enable CONFIG_PNFSD ? (That is the pNFSD Server Kernel Support)

No pNFSD Server support wasn't enabled, i recompiled and activate it

It's fine for this stage you don't need it

Quoted 7 lines

Quoted 4 lines

What platform are you using? Distro + ARCH ?

Iam experimenting with Fedora 16 (3.3.0 pnfs kernel) and arch x86_64

I use that here too

Quoted 3 lines

Thanks for your efforts
Johannes

Hope that helps. Thanks for the report we got a bug fix
Boaz
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html