Unable to get XFS, ext3, reiserfs & LVM to coexist happily

Ok, I have 2.4.17-xfs (cvs checkout today) with patches from lvm-1.0.1, applied lvm-1.0.1-2.4.17-xfs.patch then linux-2.4.11-VFS-lock.patch.

I tried creating lvm volumes & snapshots w/ an xfs filesystem.

I did run into problems (creating snapshot against my "test" volume while it was mounted caused an oops; creating snapshot volume then mounting test oopsed....) but eventually I got a volume mounted, and had a snapshot volume against it.

Overflowing the snapshot behaved well:

lvm -- giving up to snapshot /dev/test/foo on /dev/test/snap: out of space

I'll check out the oopses to see if I can figure out what's going on.

-Eric

Eric Sandeen      XFS for Linux     http://oss.sgi.com/projects/xfs
[email protected]   SGI, Inc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Well this seems consistant with what I'm seeing:
2.4.17-xfs
While source volume mounted:
+ lvm-1.0.1 upgrade
+ VFS-lock
(In that order)
* Can create ext3 snapshot? | yes
* Can mount a ext3 snapshot? | yes
* Can create resierfs snapshot? | yes
* Can mount a resierfs snapshot? | yes
* Can create xfs snapshot? | Oops
btp 1234 (lvcreate)
journal_start
ext3_dirty_inode
__mark_inode_dirty
update_atime
do_generic_file_read
generic_file_read
sys_read
system_call
Running processes (lvcreate)
* Can mount a xfs snapshot? | Cannot test
When source & snapshot mounted:
* Cause oops when snapshot overflows? |
- ext3 | OK - Stable
- resierfs | OK - Stable
- xfs | Cannot test

What does your backtrace look like? Its the ext3_dirt_inode in my backtrace thats got me. I have compiled a kernel without ext3 so will also give it a run later.

I now have a serial cable and once I work out how to use kdb with it I'll be able to capture the full kdb output.. I was wanting to be in this position now; but ran into an issue with the Linux kernel which killed a lot of time..

OT: There is an option in the linux kernel config that allows extra debugging info to be compled into the kernel. Don't use it as it stops all modules from being loaded. It took me a couple of hours to realise what was happening :-(

- --
Adrian Head

(Public Key available on request.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8NjF18ZJI8OvSkAcRAiFHAJ9oHz9ntOT+pNatBj3SxTMqw0je1gCfasWm 8zvETAseafRuSEpOaAFQkuE=
=+BBG
-----END PGP SIGNATURE-----

Can you repeat this test, but skip the ext3 snapshot creation part of it entirely (i.e. create a reiserfs snapshot and an XFS snapshot, without changing the kernel)? I'm wondering if XFS is getting an old inode that ext3 was using, but either ext3 or XFS is not clearing it, so that is why it is calling into ext3. Also, are you using ext3 on other filesystems in this computer?

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Yes there are other volumes using ext3.
/boot 20M ext3
/ 512M ext3

/usr			   512M	ext3
/var			   512M	ext3
swap			   768M
/tmp			   128M	reiserfs  LVM

/cache 512M reiserfs LVM
/chroot 512M ext3 LVM
/usr/src 4096Mreiserfs LVM
/data ~30G xfs LVM

the tests are done on /chroot for ext3,
/usr/src for resierfs & /data for xfs.

I can skip the ext3 snapshot creation part - each test is done on its own logical volume; however, in the dark dim past any of these could have had other filesystems on them.
Does this sound sensible?
1) Run tests without doing the ext3 snapshot creation, then
2) dd the entire lv that is giving problems and try again?

Thanks for your time & effort.

- --
Adrian Head

(Public Key available on request.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8NmEb8ZJI8OvSkAcRAiJGAJ9SeL5O399a56EQc2hOu3KbAlTsngCgksF+ y/yKm/SxB7lji0XMLXaAQXo=
=DRn3
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andreas or any other LVM gurus,

This is related to but not imeadiately concerned with the problem we have been discussing.

When lvcreate dies when creating an XFS snapshot it doesn't finish the snapshot creation and although lvdisplay shows that the XFS lv has an active snapshot (shows even its name) lvdisplay tells me that the snapshot doesn't exist and therefore, cannot be removed.

During the reboot the machine fails to run through the startup scripts and dies somewhere within mount.

I have found that kernels without the lvm-1.0.1 upgrade patch are able to carry on and even fix the half created snapshot whereas kernels with the lvm-1.0.1 upgrade patch are unable to deal with the problem snapshot and just die.

Are you aware of what this might be?

And if this is going to be the default behaviour in future versions of LVM how would I recover using a lvm-1.0.1 patched kernel?

Just interested

Thanks

- --
Adrian Head

(Public Key available on request.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8NnUR8ZJI8OvSkAcRAnegAJ9rayag2AQ7AMKufbL0d7PlxB3jxwCdGaQz UKinrM4ZVskDCt13J+3HAs0=
=UDmt
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This is some more info for those that are interested.

What interests me here is that the lvm-1.0.1rc4(ish) seems to be more robust then lvm-1.0.1 when dealing with a failed LV creation.

What seems to happen is that the snapshot is partially registered in the kernel. The snapshot appears in the "/proc/lvm/global" as well as in the
"/proc/lvm/VGs/HDA/LVs/". However, "/dev/HDA/SNAP" is missing and I can
generate it by doing a vgscan and a lvscan. However, this does not fix the kernel from dying upon a reboot.

The only way I have found to recover from this situation is to use an old kernel that has only lvm-1.0.1rc4(ish) - this kernel does not Oops upon boot and fixes the failed snapshot so that either it can be used later or removed correctly.

The procedure goes something like this:
1) create snapshot on an XFS partition - it fails with an Oops.
at this point I can not view or remove the failed snapshot although it appears in the correct places in /proc. The /dev/vg/lv entries are missing.
2) During the startup after a boot the machine dies after multiple Oopses and
dies finally with a Kernel panic. The kdb ouput from the start-up is appended below.
3) Reboot using an old kernel with lvm-1.0.1rc4(ish) and everything starts
fine. You can then lvremove without a problem.
4) I can then reboot into a kernel that has been upgraded to lvm-1.0.1
without a problem.

The system is based on a minimal install of RH7.2 using a SGI XFS CVS kernel.

2.4.17-xfs
+ lvm-1.0.1 upgrade (from LVM CVS)
+ VFS-lock patch

Is there any other info people need?

Once the XFS snapshot creation has failed this is what happens during the boot process after the reboot.

Unable to handle kernel NULL pointer dereference at virtual address 00000098

*pde = 00000000

Entering kdb (current=0xd75a4000, pid 152) Oops: Oops
due to oops @ 0xc0144e9e
eax = 0x00000080 ebx = 0xd8f69720 ecx = 0xc1606340 edx = 0xd75a4000 esi = 0xd6b7a000 edi = 0xd75a5f64 esp = 0xd75a5f08 eip = 0xc0144e9e ebp = 0xc1606600 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010286 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd75a5ed4 kdb> bt
EBP EIP Function(args)
0xc1606600 0xc0144e9e do_add_mount+0xfe (0xd75a5f64, 0xd6b7b000, 0x0, 0x0, 0xd6b7a000)

                               kernel .text 0xc0100000 0xc0144da0 0xc0144f60
           0xc0145137 do_mount+0x137 (0xd6b7a000, 0xd7972000, 0xd6b7b000, 
0xc0ed0000, 0xd6b79000)
                               kernel .text 0xc0100000 0xc0145000 0xc0145160
           0xc01451d4 sys_mount+0x74 (0x807f730, 0x807f748, 0x807f758, 
0xc0ed0000, 0x807f788)
                               kernel .text 0xc0100000 0xc0145160 0xc0145210
           0xc0106d1b system_call+0x33
                               kernel .text 0xc0100000 0xc0106ce8 0xc0106d20

kdb> btp 152
EBP EIP Function(args)
0xc0139b4e 0xc0144e9e do_add_mount+0xfe (0xd75a5f64, 0xd6b7b000, 0x0, 0x0, 0xd6b7a000)

                               kernel .text 0xc0100000 0xc0144da0 0xc0144f60
           0xc0145137 do_mount+0x137 (0xd6b7a000, 0xd7972000, 0xd6b7b000, 
0xc0ed0000, 0xd6b79000)
                               kernel .text 0xc0100000 0xc0145000 0xc0145160
           0xc01451d4 sys_mount+0x74 (0x807f730, 0x807f748, 0x807f758, 
0xc0ed0000, 0x807f788)
                               kernel .text 0xc0100000 0xc0145160 0xc0145210
           0xc0106d1b system_call+0x33
                               kernel .text 0xc0100000 0xc0106ce8 0xc0106d20
kdb> ps
Task Addr  Pid      Parent   [*] cpu  State Thread     Command
0xd7fe2000 00000001 00000000  1  000  stop  0xd7fe2270 init
0xc163c000 00000002 00000001  1  000  stop  0xc163c270 keventd
0xc1638000 00000003 00000000  1  000  stop  0xc1638270 ksoftirqd_CPU0
0xc1636000 00000004 00000000  1  000  stop  0xc1636270 kswapd
0xc1634000 00000005 00000000  1  000  stop  0xc1634270 bdflush
0xc1632000 00000006 00000000  1  000  stop  0xc1632270 kupdated
0xd7ec4000 00000007 00000001  1  000  stop  0xd7ec4270 mdrecoveryd
0xd7eae000 00000008 00000001  1  000  stop  0xd7eae270 raid5d
0xd7b32000 00000009 00000001  1  000  stop  0xd7b32270 kjournald
0xd7982000 00000010 00000001  1  000  stop  0xd7982270 init
0xd796e000 00000011 00000010  1  000  stop  0xd796e270 rc.sysinit
0xd7838000 00000012 00000011  1  000  stop  0xd7838270 initlog
0xd74ba000 00000111 00000001  1  000  stop  0xd74ba270 minilogd
0xd74da000 00000151 00000011  1  000  stop  0xd74da270 initlog
0xd75a4000 00000152 00000151  1  000  run   0xd75a4270*mount
0xd6e54000 00000153 00000152  1  000  stop  0xd6e54270 kjournald
0xd6e48000 00000154 00000152  1  000  stop  0xd6e48270 kjournald
0xd6e40000 00000155 00000152  1  000  stop  0xd6e40270 kjournald
0xd6d22000 00000157 00000152  1  000  run   0xd6d22270 kreiserfsd
0xd6c5c000 00000158 00000152  1  000  stop  0xd6c5c270 kjournald
0xd6b16000 00000160 00000001  1  000  stop  0xd6b16270 pagebuf_daemon
more>
kdb> id %eip
0xc0144e9e do_add_mount+0xfe:   mov    0x18(%eax),%esi
0xc0144ea1 do_add_mount+0x101:  test   %esi,%esi
0xc0144ea3 do_add_mount+0x103:  je     0xc0144eae do_add_mount+0x10e:
0xc0144ea5 do_add_mount+0x105:  incl   0x28(%esi)
0xc0144ea8 do_add_mount+0x108:  mov    0x550(%edx),%eax
0xc0144eae do_add_mount+0x10e:  mov    0xc(%eax),%ebx
0xc0144eb1 do_add_mount+0x111:  mov    %esi,%edi
0xc0144eb3 do_add_mount+0x113:  test   %ebx,%ebx
0xc0144eb5 do_add_mount+0x115:  je     0xc0144ec1 do_add_mount+0x121:
0xc0144eb7 do_add_mount+0x117:  mov    (%ebx),%eax
0xc0144eb9 do_add_mount+0x119:  test   %eax,%eax
0xc0144ebb do_add_mount+0x11b:  jne    0xc0144ebf do_add_mount+0x11f:
0xc0144ebd do_add_mount+0x11d:  ud2a
0xc0144ebf do_add_mount+0x11f:  incl   (%ebx)
0xc0144ec1 do_add_mount+0x121:  push   $0x1000
0xc0144ec6 do_add_mount+0x126:  pushl  0xc(%esp,1)
kdb> go
Oops: 0000
CPU:    0
EIP:    0010:[<c0144e9e>]    Not tainted
EFLAGS: 00010286
eax: 00000080   ebx: d8f69720   ecx: c1606340   edx: d75a4000
esi: d6b7a000   edi: d75a5f64   ebp: c1606600   esp: d75a5f08
ds: 0018   es: 0018   ss: 0018
Process mount (pid: 152, stackpage=d75a5000)
Stack: c0139b4e d76741c0 d6674000 d797200a d76741c0 00000000 d7b30700 d75a4000
       00000000 c0111e10 00000000 00000000 d7972000 d75a5f64 00000000 c0145137
       d75a5f64 d6b7b000 00000000 00000000 d6b7a000 d6b79000 00000000 d76741c0
Call Trace: [<c0139b4e>] [<c0111e10>] [<c0145137>] [<c0144fac>] [<c01451d4>]
   [<c0106d1b>]

Code: 8b 70 18 85 f6 74 09 ff 46 28 8b 82 50 05 00 00 8b 58 0c 89
 <1>Unable to handle kernel NULL pointer dereference at virtual address 

00000080 printing eip:
c0117c74
*pde = 00000000

Entering kdb (current=0xd75a4000, pid 152) Oops: Oops
due to oops @ 0xc0117c74
eax = 0x00000000 ebx = 0x00000080 ecx = 0x00000032 edx = 0xd75a4264 esi = 0xd75a4000 edi = 0x0000000b esp = 0xd75a5da4 eip = 0xc0117c74 ebp = 0xc0227590 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010202 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd75a5d70 kdb> go

Oops: 0002
CPU:    0
EIP:    0010:[<c0117c74>]    Not tainted
EFLAGS: 00010202
eax: 00000000   ebx: 00000080   ecx: 00000032   edx: d75a4264
esi: d75a4000   edi: 0000000b   ebp: c0227590   esp: d75a5da4
ds: 0018   es: 0018   ss: 0018
Process mount (pid: 152, stackpage=d75a5000)
Stack: 00000000 c0227590 c010a478 00000001 c02350fa c02c63e8 c02c4da0 00000000
       c0227590 00000001 00000018 00000018 00000001 c02c4da0 00000000 c0107411
       0000000b 000036ff 00000282 c0322e53 00000202 c0115484 00000000 c0144e9e
Call Trace: [<c010a478>] [<c0107411>] [<c0115484>] [<c0144e9e>] [<c0144e9e>]
   [<c0112176>] [<c01392ff>] [<c0135605>] [<d8f6f028>] [<c014412b>] 

[<c0111e10>]
[<c0106e0c>] [<d8f69720>] [<c0144e9e>] [<c0139b4e>] [<c0111e10>]
[<c0145137>]
[<c0144fac>] [<c01451d4>] [<c0106d1b>]

Code: ff 0b 0f 94 c0 84 c0 74 76 ff 73 0c e8 9b 8d 02 00 8b 53 18
 <1>Unable to handle kernel NULL pointer dereference at virtual address 

0000000c printing eip:
c0129cb9
*pde = 00000000

Entering kdb (current=0xd75a4000, pid 152) Oops: Oops
due to oops @ 0xc0129cb9
eax = 0x00632840 ebx = 0xc1607360 ecx = 0x00000000 edx = 0x00000000 esi = 0x00000096 edi = 0xd8ca1000 esp = 0xd75a5c0c eip = 0xc0129cb9 ebp = 0xc0227590 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010046 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd75a5bd8 kdb> go

Oops: 0000
CPU:    0
EIP:    0010:[<c0129cb9>]    Not tainted
EFLAGS: 00010046
eax: 00632840   ebx: c1607360   ecx: 00000000   edx: 00000000
esi: 00000096   edi: d8ca1000   ebp: c0227590   esp: d75a5c0c
ds: 0018   es: 0018   ss: 0018
Process mount (pid: 152, stackpage=d75a5000)
Stack: 00000000 00000400 d75a4000 0000000b c011cc0e c1607360 d8ca1000 d75a4000
       c011cc84 d75a4568 00000000 c0117cf9 d75a4000 00000002 c0227590 c010a478
       00000001 c02350fa c02c63e8 c02c4da0 00000002 c0227590 00000001 00000018
Call Trace: [<c011cc0e>] [<c011cc84>] [<c0117cf9>] [<c010a478>] [<c0107411>]
   [<c0115484>] [<c0117c74>] [<c0111e10>] [<c0117c74>] [<c0111e10>] 

[<c0112176>]
[<c0121ba8>] [<c0121821>] [<c0111e10>] [<c0106e0c>] [<c0117c74>]
[<c010a478>]
[<c0107411>] [<c0115484>] [<c0144e9e>] [<c0144e9e>] [<c0112176>]
[<c01392ff>]
[<c0135605>] [<d8f6f028>] [<c014412b>] [<c0111e10>] [<c0106e0c>]
[<d8f69720>]
[<c0144e9e>] [<c0139b4e>] [<c0111e10>] [<c0145137>] [<c0144fac>]
[<c01451d4>]
[<c0106d1b>]

Code: 2b 79 0c 89 f8 f7 73 18 89 c7 8b 41 14 89 44 b9 18 89 79 14
 <1>Unable to handle kernel paging request at virtual address 002aad20
 printing eip:

c012b40e
*pde = 00000000

Entering kdb (current=0xd74da000, pid 182) Oops: Oops
due to oops @ 0xc012b40e
eax = 0x00002bea ebx = 0xc15bd500 ecx = 0x00000003 edx = 0x002aa7a4 esi = 0xc02c7698 edi = 0x00000002 esp = 0xd74dbe3c eip = 0xc012b40e ebp = 0x00015f54 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010013 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd74dbe08 kdb> bt
EBP EIP Function(args)
0x00015f54 0xc012b40e rmqueue+0x8e

                               kernel .text 0xc0100000 0xc012b380 0xc012b540
           0xc012b73f __alloc_pages+0x3f
                               kernel .text 0xc0100000 0xc012b700 0xc012b880
kdb> btp 182
    EBP       EIP         Function(args)
0xd6af1cc0 0xc012b40e rmqueue+0x8e
                               kernel .text 0xc0100000 0xc012b380 0xc012b540
           0xc012b73f __alloc_pages+0x3f
                               kernel .text 0xc0100000 0xc012b700 0xc012b880
kdb> ps
Task Addr  Pid      Parent   [*] cpu  State Thread     Command
0xd7fe2000 00000001 00000000  1  000  stop  0xd7fe2270 init
0xc163c000 00000002 00000001  1  000  stop  0xc163c270 keventd
0xc1638000 00000003 00000000  1  000  stop  0xc1638270 ksoftirqd_CPU0
0xc1636000 00000004 00000000  1  000  stop  0xc1636270 kswapd
0xc1634000 00000005 00000000  1  000  stop  0xc1634270 bdflush
0xc1632000 00000006 00000000  1  000  stop  0xc1632270 kupdated
0xd7ec4000 00000007 00000001  1  000  stop  0xd7ec4270 mdrecoveryd
0xd7eae000 00000008 00000001  1  000  stop  0xd7eae270 raid5d
0xd7b32000 00000009 00000001  1  000  stop  0xd7b32270 kjournald
0xd7982000 00000010 00000001  1  000  stop  0xd7982270 init
0xd796e000 00000011 00000010  1  000  stop  0xd796e270 rc.sysinit
0xd7838000 00000012 00000011  1  000  stop  0xd7838270 initlog
0xd74ba000 00000111 00000001  1  000  stop  0xd74ba270 minilogd
0xd6e54000 00000153 00000001  1  000  stop  0xd6e54270 kjournald
0xd6e48000 00000154 00000001  1  000  stop  0xd6e48270 kjournald
0xd6e40000 00000155 00000001  1  000  stop  0xd6e40270 kjournald
0xd6d22000 00000157 00000001  1  000  stop  0xd6d22270 kreiserfsd
0xd6c5c000 00000158 00000001  1  000  stop  0xd6c5c270 kjournald
0xd6b16000 00000160 00000001  1  000  stop  0xd6b16270 pagebuf_daemon
0xd74da000 00000182 00000011  1  000  run   0xd74da270*pam_console_app
kdb> id %eip
0xc012b40e rmqueue+0x8e:        btc    %eax,(%edx)
0xc012b411 rmqueue+0x91:        movzbl 0x8(%esp,1),%ecx
0xc012b416 rmqueue+0x96:        mov    0xc(%esp,1),%edx
0xc012b41a rmqueue+0x9a:        mov    $0x1,%eax
0xc012b41f rmqueue+0x9f:        shl    %cl,%eax
0xc012b421 rmqueue+0xa1:        sub    %eax,0x4(%edx)
0xc012b424 rmqueue+0xa4:        mov    %ebp,(%esp,1)
0xc012b427 rmqueue+0xa7:        mov    %edi,%ebp
0xc012b429 rmqueue+0xa9:        mov    %ebp,%ecx
0xc012b42b rmqueue+0xab:        mov    $0x1,%edi
0xc012b430 rmqueue+0xb0:        shl    %cl,%edi
0xc012b432 rmqueue+0xb2:        cmp    0x8(%esp,1),%ebp
0xc012b436 rmqueue+0xb6:        jle    0xc012b49e rmqueue+0x11e:
0xc012b438 rmqueue+0xb8:        nop
0xc012b439 rmqueue+0xb9:        lea    0x0(%esi,1),%esi
0xc012b440 rmqueue+0xc0:        mov    0xc(%esp,1),%eax
kdb> go
Oops: 0002
CPU:    0
EIP:    0010:[<c012b40e>]    Not tainted
EFLAGS: 00010013
eax: 00002bea   ebx: c15bd500   ecx: 00000003   edx: 002aa7a4
esi: c02c7698   edi: 00000002   ebp: 00015f54   esp: d74dbe3c
ds: 0018   es: 0018   ss: 0018
Process pam_console_app (pid: 182, stackpage=d74db000)
Stack: d6af1cc0 00000286 00000000 c02c7668 c02c77bc 000001ff 00000000 00000000
       c012b73f c02c7668 c02c77b8 000001f0 d7720680 00000202 c1607130 000001f0
       00000000 c012b890 c0129996 00000001 000001f0 d7ed4874 00000000 d7720680
Call Trace: [<c012b73f>] [<c012b890>] [<c0129996>] [<c015cec5>] [<c0129b89>]
   [<c0157012>] [<c0141087>] [<c01393a5>] [<c01399f0>] [<c013910d>] 

[<c0139f83>]
[<c01371c4>] [<c0123b92>] [<c0106d1b>]

Code: 0f bb 02 0f b6 4c 24 08 8b 54 24 0c b8 01 00 00 00 d3 e0 29
 <1>Unable to handle kernel paging request at virtual address 002aad1c
 printing eip:

c012b40e
*pde = 00000000

Entering kdb (current=0xd796e000, pid 11) Oops: Oops
due to oops @ 0xc012b40e
eax = 0x00002bcc ebx = 0xc15b9800 ecx = 0x00000003 edx = 0x002aa7a4 esi = 0xc02c7698 edi = 0x00000002 esp = 0xd796fe54 eip = 0xc012b40e ebp = 0x00015e60 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010016 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd796fe20 kdb> bt
EBP EIP Function(args)
0x00015e60 0xc012b40e rmqueue+0x8e (0xc02c7668, 0xc02c77d8, 0x1d2, 0xc0208f4f, 0x0)

                               kernel .text 0xc0100000 0xc012b380 0xc012b540
           0xc012b73f __alloc_pages+0x3f (0xd796ffc4, 0x7, 0xcf, 0x80e498c, 
0x13)
                               kernel .text 0xc0100000 0xc012b700 0xc012b880
           0xc0106e0c error_code+0x34
                               kernel .text 0xc0100000 0xc0106dd8 0xc0106e14

Interrupt registers:
eax = 0x080e4980 ebx = 0xd796ffc4 ecx = 0x00000007 edx = 0x000000cf esi = 0x080e498c edi = 0x00000013 esp = 0x00000023 eip = 0x0000002b ebp = 0x080e498c xss = 0x00010202 xcs = 0xffffffff eflags = 0x080ac410 xds = 0xbffff838 xes = 0x080e498c origeax = 0x0000002b &regs = 0xd796ffbc Interrupt from user space, end of kernel trace
kdb> btp 11
EBP EIP Function(args)
0x00015810 0xc012b40e rmqueue+0x8e (0xc02c7668, 0xc02c77d8, 0x1d2, 0xc0208f4f, 0x0)

                               kernel .text 0xc0100000 0xc012b380 0xc012b540
           0xc012b73f __alloc_pages+0x3f (0xd796ffc4, 0x7, 0xcf, 0x80e498c, 
0x13)
                               kernel .text 0xc0100000 0xc012b700 0xc012b880
           0xc0106e0c error_code+0x34
                               kernel .text 0xc0100000 0xc0106dd8 0xc0106e14

Interrupt registers:
eax = 0x080e4980 ebx = 0xd796ffc4 ecx = 0x00000007 edx = 0x000000cf esi = 0x080e498c edi = 0x00000013 esp = 0x00000023 eip = 0x0000002b ebp = 0x080e498c xss = 0x00010202 xcs = 0xffffffff eflags = 0x080ac410 xds = 0xbffff838 xes = 0x080e498c origeax = 0x0000002b &regs = 0xd796ffbc Interrupt from user space, end of kernel trace
kdb> ps

Task Addr  Pid      Parent   [*] cpu  State Thread     Command
0xd7fe2000 00000001 00000000  1  000  stop  0xd7fe2270 init
0xc163c000 00000002 00000001  1  000  stop  0xc163c270 keventd
0xc1638000 00000003 00000000  1  000  stop  0xc1638270 ksoftirqd_CPU0
0xc1636000 00000004 00000000  1  000  stop  0xc1636270 kswapd
0xc1634000 00000005 00000000  1  000  stop  0xc1634270 bdflush
0xc1632000 00000006 00000000  1  000  stop  0xc1632270 kupdated
0xd7ec4000 00000007 00000001  1  000  stop  0xd7ec4270 mdrecoveryd
0xd7eae000 00000008 00000001  1  000  stop  0xd7eae270 raid5d
0xd7b32000 00000009 00000001  1  000  stop  0xd7b32270 kjournald
0xd7982000 00000010 00000001  1  000  stop  0xd7982270 init
0xd796e000 00000011 00000010  1  000  run   0xd796e270*rc.sysinit
0xd7838000 00000012 00000011  1  000  stop  0xd7838270 initlog
0xd74ba000 00000111 00000001  1  000  stop  0xd74ba270 minilogd
0xd6e54000 00000153 00000001  1  000  stop  0xd6e54270 kjournald
0xd6e48000 00000154 00000001  1  000  stop  0xd6e48270 kjournald
0xd6e40000 00000155 00000001  1  000  stop  0xd6e40270 kjournald
0xd6d22000 00000157 00000001  1  000  stop  0xd6d22270 kreiserfsd
0xd6c5c000 00000158 00000001  1  000  stop  0xd6c5c270 kjournald
0xd6b16000 00000160 00000001  1  000  stop  0xd6b16270 pagebuf_daemon
0xd74da000 00000183 00000011  1  000  run   0xd74da270 rc.sysinit
kdb> id %eip
0xc012b40e rmqueue+0x8e:        btc    %eax,(%edx)
0xc012b411 rmqueue+0x91:        movzbl 0x8(%esp,1),%ecx
0xc012b416 rmqueue+0x96:        mov    0xc(%esp,1),%edx
0xc012b41a rmqueue+0x9a:        mov    $0x1,%eax
0xc012b41f rmqueue+0x9f:        shl    %cl,%eax
0xc012b421 rmqueue+0xa1:        sub    %eax,0x4(%edx)
0xc012b424 rmqueue+0xa4:        mov    %ebp,(%esp,1)
0xc012b427 rmqueue+0xa7:        mov    %edi,%ebp
0xc012b429 rmqueue+0xa9:        mov    %ebp,%ecx
0xc012b42b rmqueue+0xab:        mov    $0x1,%edi
0xc012b430 rmqueue+0xb0:        shl    %cl,%edi
0xc012b432 rmqueue+0xb2:        cmp    0x8(%esp,1),%ebp
0xc012b436 rmqueue+0xb6:        jle    0xc012b49e rmqueue+0x11e:
0xc012b438 rmqueue+0xb8:        nop
0xc012b439 rmqueue+0xb9:        lea    0x0(%esi,1),%esi
0xc012b440 rmqueue+0xc0:        mov    0xc(%esp,1),%eax
kdb> go
Oops: 0002
CPU:    0
EIP:    0010:[<c012b40e>]    Not tainted
EFLAGS: 00010016
eax: 00002bcc   ebx: c15b9800   ecx: 00000003   edx: 002aa7a4
esi: c02c7698   edi: 00000002   ebp: 00015e60   esp: d796fe54
ds: 0018   es: 0018   ss: 0018
Process rc.sysinit (pid: 11, stackpage=d796f000)
Stack: 00015810 00000282 00000000 c02c7668 c02c77dc 000001ff 00000000 080e498c
       c012b73f c02c7668 c02c77d8 000001d2 c0208f4f 00000000 c15950c0 16543065
       080e498c c0122505 c02c7668 c02c77bc 000001ff 00000000 080e498c 00000001
Call Trace: [<c012b73f>] [<c0208f4f>] [<c0122505>] [<c0122ae6>] [<c0122b7a>]
   [<c020a341>] [<c0111f8e>] [<c012b73f>] [<c01148b7>] [<c01309c0>] 

[<c0111e10>]
[<c0106e0c>]

Code: 0f bb 02 0f b6 4c 24 08 8b 54 24 0c b8 01 00 00 00 d3 e0 29
 <1>Unable to handle kernel paging request at virtual address 002aad48
 printing eip:

c012b2fe
*pde = 00000000

Entering kdb (current=0xd7982000, pid 10) Oops: Oops
due to oops @ 0xc012b2fe
eax = 0xc02c76f8 ebx = 0xc02c7668 ecx = 0xc02c7698 edx = 0x002aa7a4 esi = 0x0001696c edi = 0x00002d2d esp = 0xd7983f68 eip = 0xc012b2fe ebp = 0xc15e5b80 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010087 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd7983f34 kdb> bt
EBP EIP Function(args)
0xc15e5b80 0xc012b2fe __free_pages_ok+0x16e

                               kernel .text 0xc0100000 0xc012b190 0xc012b380
kdb> btp 10
    EBP       EIP         Function(args)
0xc1040000 0xc012b2fe __free_pages_ok+0x16e
                               kernel .text 0xc0100000 0xc012b190 0xc012b380
kdb> ps
Task Addr  Pid      Parent   [*] cpu  State Thread     Command
0xd7fe2000 00000001 00000000  1  000  stop  0xd7fe2270 init
0xc163c000 00000002 00000001  1  000  stop  0xc163c270 keventd
0xc1638000 00000003 00000000  1  000  stop  0xc1638270 ksoftirqd_CPU0
0xc1636000 00000004 00000000  1  000  stop  0xc1636270 kswapd
0xc1634000 00000005 00000000  1  000  stop  0xc1634270 bdflush
0xc1632000 00000006 00000000  1  000  stop  0xc1632270 kupdated
0xd7ec4000 00000007 00000001  1  000  stop  0xd7ec4270 mdrecoveryd
0xd7eae000 00000008 00000001  1  000  stop  0xd7eae270 raid5d
0xd7b32000 00000009 00000001  1  000  stop  0xd7b32270 kjournald
0xd7982000 00000010 00000001  1  000  stop  0xd7982270*init
0xd7838000 00000012 00000001  1  000  stop  0xd7838270 initlog
0xd74ba000 00000111 00000001  1  000  stop  0xd74ba270 minilogd
0xd6e54000 00000153 00000001  1  000  stop  0xd6e54270 kjournald
0xd6e48000 00000154 00000001  1  000  stop  0xd6e48270 kjournald
0xd6e40000 00000155 00000001  1  000  stop  0xd6e40270 kjournald
0xd6d22000 00000157 00000001  1  000  stop  0xd6d22270 kreiserfsd
0xd6c5c000 00000158 00000001  1  000  stop  0xd6c5c270 kjournald
0xd6b16000 00000160 00000001  1  000  stop  0xd6b16270 pagebuf_daemon
0xd74da000 00000183 00000001  1  000  run   0xd74da270 rc.sysinit
kdb> id %eip
0xc012b2fe __free_pages_ok+0x16e:       btc    %edi,(%edx)
0xc012b301 __free_pages_ok+0x171:       sbb    %eax,%eax
0xc012b303 __free_pages_ok+0x173:       test   %eax,%eax
0xc012b305 __free_pages_ok+0x175:       jne    0xc012b260 
__free_pages_ok+0xd0: 0xc012b30b __free_pages_ok+0x17b:       mov    
(%esp,1),%edx
0xc012b30e __free_pages_ok+0x17e:       mov    0x4(%esp,1),%ecx
0xc012b312 __free_pages_ok+0x182:       shl    $0x6,%esi
0xc012b315 __free_pages_ok+0x185:       lea    (%edx,%esi,1),%eax
0xc012b318 __free_pages_ok+0x188:       mov    (%ecx),%edx
0xc012b31a __free_pages_ok+0x18a:       mov    %eax,0x4(%edx)
0xc012b31d __free_pages_ok+0x18d:       mov    %edx,(%eax)
0xc012b31f __free_pages_ok+0x18f:       mov    %ecx,0x4(%eax)
0xc012b322 __free_pages_ok+0x192:       mov    %eax,(%ecx)
0xc012b324 __free_pages_ok+0x194:       pushl  0x8(%esp,1)
0xc012b328 __free_pages_ok+0x198:       popf
0xc012b329 __free_pages_ok+0x199:       jmp    0xc012b36c 
__free_pages_ok+0x1dc:kdb> go
Oops: 0002
CPU:    0
EIP:    0010:[<c012b2fe>]    Not tainted
EFLAGS: 00010087
eax: c02c76f8   ebx: c02c7668   ecx: c02c7698   edx: 002aa7a4
esi: 0001696c   edi: 00002d2d   ebp: c15e5b80   esp: d7983f68
ds: 0018   es: 0018   ss: 0018
Process init (pid: 10, stackpage=d7983000)
Stack: c1040000 c02c7698 00000213 fffffffc d796e000 d7982000 0000000b 0000000b
       c01180c6 d796e000 00000000 d7982000 00000000 00000000 00000000 d7982000
       d79820bc d79820bc d7982000 00000000 00000000 bffff7f8 c0106d1b 0000000b
Call Trace: [<c01180c6>] [<c0106d1b>]

Code: 0f bb 3a 19 c0 85 c0 0f 85 55 ff ff ff 8b 14 24 8b 4c 24 04
 <1>Unable to handle kernel paging request at virtual address 002aad48
 printing eip:

c012b2fe
*pde = 00000000

Entering kdb (current=0xd7fe2000, pid 1) Oops: Oops
due to oops @ 0xc012b2fe
eax = 0xc02c76f8 ebx = 0xc02c7668 ecx = 0xc02c7698 edx = 0x002aa7a4 esi = 0x00016980 edi = 0x00002d30 esp = 0xd7fe3f68 eip = 0xc012b2fe ebp = 0xc15e6080 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010087 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd7fe3f34 kdb> bt
EBP EIP Function(args)
0xc15e6080 0xc012b2fe __free_pages_ok+0x16e

                               kernel .text 0xc0100000 0xc012b190 0xc012b380
kdb> btp 1
    EBP       EIP         Function(args)
0xc1040000 0xc012b2fe __free_pages_ok+0x16e
                               kernel .text 0xc0100000 0xc012b190 0xc012b380
kdb> ps
Task Addr  Pid      Parent   [*] cpu  State Thread     Command
0xd7fe2000 00000001 00000000  1  000  stop  0xd7fe2270*init
0xc163c000 00000002 00000001  1  000  stop  0xc163c270 keventd
0xc1638000 00000003 00000000  1  000  stop  0xc1638270 ksoftirqd_CPU0
0xc1636000 00000004 00000000  1  000  stop  0xc1636270 kswapd
0xc1634000 00000005 00000000  1  000  stop  0xc1634270 bdflush
0xc1632000 00000006 00000000  1  000  stop  0xc1632270 kupdated
0xd7ec4000 00000007 00000001  1  000  stop  0xd7ec4270 mdrecoveryd
0xd7eae000 00000008 00000001  1  000  stop  0xd7eae270 raid5d
0xd7b32000 00000009 00000001  1  000  stop  0xd7b32270 kjournald
0xd7838000 00000012 00000001  1  000  run   0xd7838270 initlog
0xd74ba000 00000111 00000001  1  000  stop  0xd74ba270 minilogd
0xd6e54000 00000153 00000001  1  000  stop  0xd6e54270 kjournald
0xd6e48000 00000154 00000001  1  000  stop  0xd6e48270 kjournald
0xd6e40000 00000155 00000001  1  000  stop  0xd6e40270 kjournald
0xd6d22000 00000157 00000001  1  000  stop  0xd6d22270 kreiserfsd
0xd6c5c000 00000158 00000001  1  000  stop  0xd6c5c270 kjournald
0xd6b16000 00000160 00000001  1  000  stop  0xd6b16270 pagebuf_daemon
0xd74da000 00000183 00000001  1  000  run   0xd74da270 rc.sysinit
kdb> id %eip
0xc012b2fe __free_pages_ok+0x16e:       btc    %edi,(%edx)
0xc012b301 __free_pages_ok+0x171:       sbb    %eax,%eax
0xc012b303 __free_pages_ok+0x173:       test   %eax,%eax
0xc012b305 __free_pages_ok+0x175:       jne    0xc012b260 
__free_pages_ok+0xd0: 0xc012b30b __free_pages_ok+0x17b:       mov    
(%esp,1),%edx
0xc012b30e __free_pages_ok+0x17e:       mov    0x4(%esp,1),%ecx
0xc012b312 __free_pages_ok+0x182:       shl    $0x6,%esi
0xc012b315 __free_pages_ok+0x185:       lea    (%edx,%esi,1),%eax
0xc012b318 __free_pages_ok+0x188:       mov    (%ecx),%edx
0xc012b31a __free_pages_ok+0x18a:       mov    %eax,0x4(%edx)
0xc012b31d __free_pages_ok+0x18d:       mov    %edx,(%eax)
0xc012b31f __free_pages_ok+0x18f:       mov    %ecx,0x4(%eax)
0xc012b322 __free_pages_ok+0x192:       mov    %eax,(%ecx)
0xc012b324 __free_pages_ok+0x194:       pushl  0x8(%esp,1)
0xc012b328 __free_pages_ok+0x198:       popf
0xc012b329 __free_pages_ok+0x199:       jmp    0xc012b36c 
__free_pages_ok+0x1dc:kdb> go
Oops: 0002
CPU:    0
EIP:    0010:[<c012b2fe>]    Not tainted
EFLAGS: 00010087
eax: c02c76f8   ebx: c02c7668   ecx: c02c7698   edx: 002aa7a4
esi: 00016980   edi: 00002d30   ebp: c15e6080   esp: d7fe3f68
ds: 0018   es: 0018   ss: 0018
Process init (pid: 1, stackpage=d7fe3000)
Stack: c1040000 c02c7698 00000217 fffffffc d7982000 d7fe2000 0000000a ffffffff
       c01180c6 d7982000 00000000 d7fe2000 00000000 00000000 00000000 d7fe2000
       d7fe20bc d7fe20bc d7fe2000 00000000 bffff52c bffff50c c0106d1b ffffffff
Call Trace: [<c01180c6>] [<c0106d1b>]

Code: 0f bb 3a 19 c0 85 c0 0f 85 55 ff ff ff 8b 14 24 8b 4c 24 04
 <0>Kernel panic: Attempted to kill init!

Entering kdb (current=0xd7fe2000, pid 1) due to panic kdb> bt
Stack is not in kernel space, backtrace not available kdb>

- --
Adrian Head

(Public Key available on request.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8N+mN8ZJI8OvSkAcRAoKyAJ952XozB1v8dSS8A8k8sdW/4iYDIQCfZjBR Nizi+BpBBeeAzCty9RSQo5g=
=jW02
-----END PGP SIGNATURE-----

Hmm, no I'm not worried about old filesystems on the disk. I thought maybe you were running the tests on the same LV, and creating new filesystems on a single LV between the tests. What I'm worried about is old inodes being kept in memory between the tests. If the tests are being run on separate LVs, then that is not a possibility.

I guess what is needed is a real oops report from the ext3 problem. Maybe Andrew or Stephen can figure it out.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

In the process of trying to fix this issue I have been playing around with a patch from the great SGI XFS developers. It certainly fixes the problems I was having but I'm now trying to help clean the patch up.

The problem now is that when I try to mount a ext3 snapshot using:
mount /dev/HDA/SNAP /mnt/snapshot
The kernel mounts the snapshot as ext2. Is was not happening before the patch and I was wondering if anyone thinks this is a problem. As ext3 is backward compatible with ext2 there doesn't seem to be any problems during my testing with respect to standard file operations. I can mount the snapshot as ext3 if I use:
mount -t ext3 /dev/HDA/SNAP /mnt/snapshot

The next problem is that when the ext3 snapshot (mounted as ext3) overflows lvm generates an error in the logs:
lvm -- giving up to snapshot /dev/HDA/XFS on /dev/HDA/SNAP: out of space lvm - lvm_map: ll_rw_blk for inactive LV /dev/HDA/SNAP

The 1st line I understand but the 2nd I don't. I was hoping that some LVM guru/developer could explain the 2nd line and what it means.

The last problem is that in only some of my tests when I "ls" the overflowed snapshot "ls" complains about missing directories and files, "echo *" shows what use to be there and ext3 generates errors in the logs:
EXT3-fs error (device lvm(58,5)): ext3_readdir" directory #2 contains a hole at offset 0

I was hoping that a ext2/3 guru/developer could explain this error and what it means.

Thanks for everybodies time and effort.

Adrian Head

(Public Key available on request.)

When a snapshot gets full it can no longer be valid so LVM makes it inactive and fails any further I/O to it.

Because there's no more space for copy-on-write operations into the snapshot there's no way the filesystem can be consistent. OK this isn't that nice but it's arguably better than returning corrupted data.

patrick

Thanks Patrick for you response. If this is the correct behaviour then everything is fine. I just asked so I could find out. :-)

Everything looks mighty fine. Thanks everyone for your time and effort.

Adrian Head

(Public Key available on request.)

this depends on the order the mount porgram tries to autodetect a filesystem if the luser does not specify it. I don't have mount sources at hand but AFAIR it reads /etc/filesystems, /proc/filesystems and then tries an hardcoded list. you can modify /etc/filesystems to change the probe order.
[you already got answers to the other issues]

L.

Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Well, given that the snapshot is now broken and LVM is refusing I/O on it, what do you expect the filesystem to do? This is what LVM _intends_ to happen, because the snapshot can no longer hold all of the data to keep it consistent.

The ext3 error is reporting that it tried to read the directory (root = #2) and it got nothing back. The reason you see some files and not others is because those blocks happen to still be in the cache, and it does not go to LVM to try and re-read them. Eventually, as the memory is needed for other things, you will no longer be able to read anything.

I suppose it would be possible to have LVM do an "unmount" of any fs using a bad snapshot device from within the kernel, but whether people actually want that to happen is another question entirely.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

If this is what is considered normal operation then everything is fine. Thats why I decided to ask - to find out :-).

From my previous tests the behaviour I was observing was that once the snapshot had overflowed a "ls" on the snapshot showed nothing. This still appears to be the behaviour of resierfs.

Cool - if this is standard behaviour then everything seems fixed :-)

Adrian Head

(Public Key available on request.)

No, ext3 functions do not show up for me when it oopses on snapshot creation:

kdb> bt
EBP EIP Function(args)
0xc1c2bf78 0xc013649e path_init+0x36 (0xc1c2a000)

                               kernel .text 0xc0100000 0xc0136468 0xc013659c
0xc1c2bf90 0xc01366cf __user_walk+0x2f (0xc1c2a000, 0x804f1bc)
                               kernel .text 0xc0100000 0xc01366a0 0xc01366f8
0xc1c2bfbc 0xc013381e sys_stat64+0x1a (0x8052474, 0xbfffe980, 0x40196154, 0x804f1bc, 0x3)
                               kernel .text 0xc0100000 0xc0133804 0xc0133874
           0xc0106c5b system_call+0x33
                               kernel .text 0xc0100000 0xc0106c28 0xc0106c60

It's trying to stat64 /dev/sda2, one of my lvm partitions. I dunno why this blows up. :/

The code has some fastcalls & inlines around here, though, so this may not be the most accurate.

Still looking...

-Eric

Eric Sandeen      XFS for Linux     http://oss.sgi.com/projects/xfs
[email protected]   SGI, Inc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This is the full output from kdb when trying to create a snapshot on an XFS volume for me.

What I was going to do next was follow Andreas Dilger's sugestion and try again.

Entering kdb (current=0xc7228000, pid 1435) Oops: Oops
due to oops @ 0xc015c446
eax = 0x748b5356 ebx = 0xc01a1550 ecx = 0xd7894040 edx = 0x00000000 esi = 0xc7228000 edi = 0xd7894040 esp = 0xc7229eac eip = 0xc015c446 ebp = 0xd7ed4800 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010286 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xc7229e78 kdb> bt
EBP EIP Function(args)
0xd7ed4800 0xc015c446 journal_start+0x36 (0xd7ed4800, 0x1, 0x40018000, 0xc7229f5c, 0x2)

                               kernel .text 0xc0100000 0xc015c410 0xc015c4f0
           0xc01571a8 ext3_dirty_inode+0x58 (0xd7894040)
                               kernel .text 0xc0100000 0xc0157150 0xc0157250
           0xc0141b1e __mark_inode_dirty+0x2e (0xd7894040, 0x1)
                               kernel .text 0xc0100000 0xc0141af0 0xc0141b70
           0xc0143001 update_atime+0x51 (0xd7894040, 0x6d6, 0x1, 0x0, 0x6d6)
                               kernel .text 0xc0100000 0xc0142fb0 0xc0143010
           0xc01254ac do_generic_file_read+0x40c (0xd5fc7740, 0xd5fc7760, 
0xc7229f5c, 0xc0125690, 0x6d6)
                               kernel .text 0xc0100000 0xc01250a0 0xc01254c0
           0xc012576a generic_file_read+0x7a (0xd5fc7740, 0x40018000, 0x1000, 
0xd5fc7760, 0x0)
                               kernel .text 0xc0100000 0xc01256f0 0xc0125810
           0xc0130cf5 sys_read+0x95 (0x4, 0x40018000, 0x1000, 0x8086ab0, 
0x40018000)
                               kernel .text 0xc0100000 0xc0130c60 0xc0130d30
           0xc0106d1b system_call+0x33
                               kernel .text 0xc0100000 0xc0106ce8 0xc0106d20

kdb> btp 1435
EBP EIP Function(args)
0xc01a1550 0xc015c446 journal_start+0x36 (0xd7ed4800, 0x1, 0x40018000, 0xc7229f5c, 0x2)

                               kernel .text 0xc0100000 0xc015c410 0xc015c4f0
           0xc01571a8 ext3_dirty_inode+0x58 (0xd7894040)
                               kernel .text 0xc0100000 0xc0157150 0xc0157250
           0xc0141b1e __mark_inode_dirty+0x2e (0xd7894040, 0x1)
                               kernel .text 0xc0100000 0xc0141af0 0xc0141b70
           0xc0143001 update_atime+0x51 (0xd7894040, 0x6d6, 0x1, 0x0, 0x6d6)
                               kernel .text 0xc0100000 0xc0142fb0 0xc0143010
           0xc01254ac do_generic_file_read+0x40c (0xd5fc7740, 0xd5fc7760, 
0xc7229f5c, 0xc0125690, 0x6d6)
                               kernel .text 0xc0100000 0xc01250a0 0xc01254c0
           0xc012576a generic_file_read+0x7a (0xd5fc7740, 0x40018000, 0x1000, 
0xd5fc7760, 0x0)
                               kernel .text 0xc0100000 0xc01256f0 0xc0125810
           0xc0130cf5 sys_read+0x95 (0x4, 0x40018000, 0x1000, 0x8086ab0, 
0x40018000)
                               kernel .text 0xc0100000 0xc0130c60 0xc0130d30
           0xc0106d1b system_call+0x33
                               kernel .text 0xc0100000 0xc0106ce8 0xc0106d20
kdb> ps
Task Addr  Pid      Parent   [*] cpu  State Thread     Command
0xd7fe2000 00000001 00000000  1  000  stop  0xd7fe2270 init
0xc163c000 00000002 00000001  1  000  stop  0xc163c270 keventd
0xc1638000 00000003 00000000  1  000  stop  0xc1638270 ksoftirqd_CPU0
0xc1636000 00000004 00000000  1  000  stop  0xc1636270 kswapd
0xc1634000 00000005 00000000  1  000  stop  0xc1634270 bdflush
0xc1632000 00000006 00000000  1  000  stop  0xc1632270 kupdated
0xd7ec4000 00000007 00000001  1  000  stop  0xd7ec4270 mdrecoveryd
0xd7eae000 00000008 00000001  1  000  stop  0xd7eae270 raid5d
0xd7b32000 00000009 00000001  1  000  stop  0xd7b32270 kjournald
0xd72a6000 00000153 00000001  1  000  stop  0xd72a6270 kjournald
0xd729c000 00000154 00000001  1  000  stop  0xd729c270 kjournald
0xd7292000 00000155 00000001  1  000  stop  0xd7292270 kjournald
0xd7174000 00000157 00000001  1  000  stop  0xd7174270 kreiserfsd
0xd70ac000 00000158 00000001  1  000  stop  0xd70ac270 kjournald
0xd6f68000 00000160 00000001  1  000  stop  0xd6f68270 pagebuf_daemon
0xd6250000 00000525 00000001  1  000  stop  0xd6250270 dhcpcd
0xd6c56000 00000661 00000001  1  000  stop  0xd6c56270 syslogd
0xd623a000 00000666 00000001  1  000  stop  0xd623a270 klogd
0xd619e000 00000686 00000001  1  000  stop  0xd619e270 portmap
0xd60ac000 00000714 00000001  1  000  stop  0xd60ac270 rpc.statd
0xd62d6000 00000827 00000001  1  000  stop  0xd62d6270 crond
more>
0xd5f3a000 00000863 00000001  1  000  stop  0xd5f3a270 atd
0xd6a80000 00000870 00000001  1  000  stop  0xd6a80270 login
0xd5f5a000 00000871 00000001  1  000  stop  0xd5f5a270 login
0xd5f56000 00000872 00000001  1  000  stop  0xd5f56270 mingetty
0xd6c9c000 00000873 00000001  1  000  stop  0xd6c9c270 mingetty
0xd5f6a000 00000874 00000001  1  000  stop  0xd5f6a270 mingetty
0xd6c9a000 00000875 00000001  1  000  stop  0xd6c9a270 mingetty
0xd6338000 00000876 00000001  1  000  stop  0xd6338270 login
0xd739c000 00000879 00000876  1  000  stop  0xd739c270 bash
0xd5d86000 00000925 00000870  1  000  stop  0xd5d86270 bash
0xd580c000 00000977 00000871  1  000  stop  0xd580c270 bash
0xc7228000 00001435 00000925  1  000  run   0xc7228270*lvcreate
kdb> id %eip
0xc015c446 journal_start+0x36:  cmp    %ebp,(%eax)
0xc015c448 journal_start+0x38:  je     0xc015c46d journal_start+0x5d:
0xc015c44a journal_start+0x3a:  push   $0xc02309e0
0xc015c44f journal_start+0x3f:  push   $0xe1
0xc015c454 journal_start+0x44:  push   $0xc022becf
0xc015c459 journal_start+0x49:  push   $0xc022b6a0
0xc015c45e journal_start+0x4e:  push   $0xc022e6a0
0xc015c463 journal_start+0x53:  call   0xc0115380 printk:
0xc015c468 journal_start+0x58:  ud2a
0xc015c46a journal_start+0x5a:  add    $0x14,%esp
0xc015c46d journal_start+0x5d:  incl   0x8(%ebx)
0xc015c470 journal_start+0x60:  jmp    0xc015c4e0 journal_start+0xd0:
0xc015c472 journal_start+0x62:  push   $0x1
0xc015c474 journal_start+0x64:  push   $0xf0
0xc015c479 journal_start+0x69:  push   $0x14
0xc015c47b journal_start+0x6b:  push   $0xc022b6a0
kdb> go
Oops: 0000
CPU:    0
EIP:    0010:[<c015c446>]    Not tainted
EFLAGS: 00010286
eax: 748b5356   ebx: c01a1550   ecx: d7894040   edx: 00000000
esi: c7228000   edi: d7894040   ebp: d7ed4800   esp: c7229eac
ds: 0018   es: 0018   ss: 0018
Process lvcreate (pid: 1435, stackpage=c7229000)
Stack: c01a1550 c01a1550 ffffffe2 d7894040 00000000 c01571a8 d7ed4800 00000001
       40018000 c7229f5c 00000002 00000018 d7894040 d7b41000 00000001 c0141b1e
       d7894040 3c36dd6f d5fc7760 d78940f0 c0143001 d7894040 00000001 00000000
Call Trace: [<c01a1550>] [<c01a1550>] [<c01571a8>] [<c0141b1e>] [<c0143001>]
   [<c01254ac>] [<c012576a>] [<c0125690>] [<c0130cf5>] [<c0106d1b>]

Code: 39 28 74 23 68 e0 09 23 c0 68 e1 00 00 00 68 cf be 22 c0 68

- --
Adrian Head

(Public Key available on request.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8NmkN8ZJI8OvSkAcRAk6FAJ0SKQPujP4BQbjs0tZprVhAyDwehQCfcjxA lONpZIiIHSojcBPRkyNFfvA=
=k23i
-----END PGP SIGNATURE-----

I've forwarded this to ext2-devel so the ext3 folks see it.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Ok, I've done a bit of kdb sleuthing over here... it looks like it's blowing up shortly after lvm_snapshot_COW, although for some reason kdb says the backtrace is from sys_stat64 after the oops. On the other hand, setting a breakpoint at
lvm_snapshot_COW, it oopses just a couple steps later. Here's the backtrace out of lvm_snapshot_COW. It gets there from lvm_do_lv_creat via unlockfs; does this make sense? I didn't expect to get into lvm_snapshot_COW until after the snapshot volume was created...

kdb> bp lvm_snapshot_COW
Instruction(i) BP #0 at 0xc8848ab4 ([lvm-mod]lvm_snapshot_COW)
is enabled globally adjust 1
kdb> go
Instruction(i) breakpoint #0 at 0xc8848ab4 (adjusted)
0xc8848ab4 lvm_snapshot_COW: int3

Entering kdb (current=0xc5ca0000, pid 1063) due to Breakpoint @ 0xc8848ab4 kdb> bt
EBP EIP Function(args)
0xc5ca1718 0xc8848ab4 [lvm-mod]lvm_snapshot_COW (0x802, 0x10178, 0x10178, 0x10178, 0xc5e8f000)

                               lvm-mod .text 0xc8844060 0xc8848ab4
0xc8848fc0
           0xc884538b [lvm-mod]__remap_snapshot+0x5b (0x802, 0x10178,
0x10178, 0xc5b10a00, 0xc5e8f000)
                               lvm-mod .text 0xc8844060 0xc8845330
0xc88453b8
0xc5ca179c 0xc8845760 [lvm-mod]lvm_map+0x3a8 (0xc5777900, 0x1)
                               lvm-mod .text 0xc8844060 0xc88453b8
0xc88457e8
0xc5ca17ac 0xc88457f8 [lvm-mod]lvm_make_request_fn+0x10 (0xc039e028, 0x1,
0xc5777900)
                               lvm-mod .text 0xc8844060 0xc88457e8
0xc8845808
0xc5ca17d0 0xc01a004b generic_make_request+0xa7 (0x1, 0xc5777900)
                               kernel .text 0xc0100000 0xc019ffa4
0xc01a00d0
0xc5ca182c 0xc882caa2 [pagebuf]_pagebuf_page_io+0x24a (0xc116c400,
0xc5caa080, 0x1, 0x0, 0x3a00)
                               pagebuf .text 0xc882b060 0xc882c858
0xc882cb1c
0xc5ca1888 0xc882cc6b [pagebuf]_page_buf_page_apply+0x14f (0xc5caa080,
0x0, 0x0, 0xc116c400, 0x600)
                               pagebuf .text 0xc882b060 0xc882cb1c
0xc882cc78
0xc5ca18c8 0xc882d0a8 [pagebuf]pagebuf_segment_apply+0x98 (0xc882cb1c,
0xc5caa080)
                               pagebuf .text 0xc882b060 0xc882d010
0xc882d0f0
0xc5ca1908 0xc882cd64 [pagebuf]pagebuf_iorequest+0xec (0xc5caa080)
                               pagebuf .text 0xc882b060 0xc882cc78
0xc882cdb8
0xc5ca1914 0xc88c977a [xfs]xfsbdstrat+0x2a (0xc13cfc00, 0xc5caa080)
                               xfs .text 0xc8866060 0xc88c9750 0xc88c978c
0xc5ca1934 0xc88b20a8 [xfs]xfs_unmountfs_writesb+0xc4 (0xc13cfc00)
more>
                               xfs .text 0xc8866060 0xc88b1fe4 0xc88b2108
0xc5ca1944 0xc889ef4d [xfs]xfs_fs_thaw+0xd (0xc13cfc00)
                               xfs .text 0xc8866060 0xc889ef40 0xc889ef60
0xc5ca1ce0 0xc88c55b0 [xfs]xfs_ioctl+0x1684 (0xc5ca6058, 0xc5b11440, 0x0,
0xc0045878, 0x0)
                               xfs .text 0xc8866060 0xc88c3f2c 0xc88c5660
0xc5ca1d08 0xc88caabd [xfs]linvfs_unfreeze_fs+0x41 (0xc13cf400)
                               xfs .text 0xc8866060 0xc88caa7c 0xc88caacc
0xc5ca1d18 0xc0131469 unlockfs+0x31 (0x3a00)
                               kernel .text 0xc0100000 0xc0131438
0xc013148c
0xc5ca1da8 0xc8847174 [lvm-mod]lvm_do_lv_create+0x7fc (0x0, 0xc884ce60,
0xc5ca1dec, 0xc6482cc0)
                               lvm-mod .text 0xc8844060 0xc8846978
0xc88471ac
0xc5ca1f90 0xc8844900 [lvm-mod]lvm_chr_ioctl+0x644 (0xc5b11800,
0xc6482cc0, 0x4004fe20, 0xbfffec60, 0xc5ca0000)
                               lvm-mod .text 0xc8844060 0xc88442bc
0xc8844a0c
0xc5ca1fbc 0xc01393f4 sys_ioctl+0x174 (0x4, 0x4004fe20, 0xbfffec60,
0x804f1bc, 0xbffff940)
                               kernel .text 0xc0100000 0xc0139280
0xc0139410
           0xc0106c5b system_call+0x33
                               kernel .text 0xc0100000 0xc0106c28

0xc0106c60

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This is the kdb output from trying to create a snapshot of an XFS volume when I have removed ext3 from my kernel. (For clarifcation - in all my previous tests ext3 was compiled into my kernel - in this test I did not select ext3 at all in menuconfig)

The only other thing I have noticed is that when the original XFS logical volume is unmounted snapshot creation is fine. I'm not sure if this works on the other tests I have tried so I will go back and redo them.

id %eip is weird. What does this really mean? looks like the instructions point nowhere. Am I correct?

Entering kdb (current=0xd600e000, pid 940) Oops: Oops
due to oops @ 0xb800
eax = 0xffffffff ebx = 0xd600e000 ecx = 0x0000b800 edx = 0xc018fd25 esi = 0x00000008 edi = 0xd600e000 esp = 0xd600ff0c eip = 0x0000b800 ebp = 0xd600ff30 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010086 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd600fed8 kdb> bt
EBP EIP Function(args)
0xd600ff30 0x0000b800 <unknown>+0xb800 (0x1)

                               kernel <unknown> 0x0 0x0 0x0
           0xc011ce83 dequeue_signal+0x43 (0xd600e560, 0xd600ff30, 
0xd600e560, 0xd600ffc4, 0xc01392ff)
                               kernel .text 0xc0100000 0xc011ce40 0xc011cef0
           0xc01069b9 do_signal+0x59 (0x11, 0xbfffec40, 0xbfffebb0, 0x8, 0x11)
                               kernel .text 0xc0100000 0xc0106960 0xc0106c00
           0xc0106d54 signal_return+0x14
                               kernel .text 0xc0100000 0xc0106d40 0xc0106d58
kdb> btp 940
    EBP       EIP         Function(args)
0xc011ce83 0x0000b800 <unknown>+0xb800 (0x1)
                               kernel <unknown> 0x0 0x0 0x0
           0xc011ce83 dequeue_signal+0x43 (0xd600e560, 0xd600ff30, 
0xd600e560, 0xd600ffc4, 0xc01392ff)
                               kernel .text 0xc0100000 0xc011ce40 0xc011cef0
           0xc01069b9 do_signal+0x59 (0x11, 0xbfffec40, 0xbfffebb0, 0x8, 0x11)
                               kernel .text 0xc0100000 0xc0106960 0xc0106c00
           0xc0106d54 signal_return+0x14
                               kernel .text 0xc0100000 0xc0106d40 0xc0106d58
kdb> ps
Task Addr  Pid      Parent   [*] cpu  State Thread     Command
0xd7fe2000 00000001 00000000  1  000  stop  0xd7fe2270 init
0xc163c000 00000002 00000001  1  000  stop  0xc163c270 keventd
0xc1638000 00000003 00000000  1  000  stop  0xc1638270 ksoftirqd_CPU0
0xc1636000 00000004 00000000  1  000  stop  0xc1636270 kswapd
0xc1634000 00000005 00000000  1  000  stop  0xc1634270 bdflush
0xc1632000 00000006 00000000  1  000  stop  0xc1632270 kupdated
0xd7ec4000 00000007 00000001  1  000  stop  0xd7ec4270 mdrecoveryd
0xd7eae000 00000008 00000001  1  000  stop  0xd7eae270 raid5d
0xd7228000 00000156 00000001  1  000  stop  0xd7228270 kreiserfsd
0xd7022000 00000159 00000001  1  000  stop  0xd7022270 pagebuf_daemon
0xd630c000 00000522 00000001  1  000  stop  0xd630c270 dhcpcd
0xd6fce000 00000658 00000001  1  000  stop  0xd6fce270 syslogd
0xd630a000 00000663 00000001  1  000  stop  0xd630a270 klogd
0xd61a2000 00000683 00000001  1  000  stop  0xd61a2270 portmap
0xd617c000 00000711 00000001  1  000  stop  0xd617c270 rpc.statd
0xd6108000 00000823 00000001  1  000  stop  0xd6108270 crond
0xd651c000 00000860 00000001  1  000  stop  0xd651c270 atd
0xd7986000 00000867 00000001  1  000  stop  0xd7986270 login
0xd627a000 00000868 00000001  1  000  stop  0xd627a270 mingetty
0xd6194000 00000869 00000001  1  000  stop  0xd6194270 mingetty
0xd613e000 00000870 00000001  1  000  stop  0xd613e270 mingetty
more>
0xd6400000 00000871 00000001  1  000  stop  0xd6400270 mingetty
0xd60e4000 00000872 00000001  1  000  stop  0xd60e4270 mingetty
0xd6284000 00000873 00000001  1  000  stop  0xd6284270 mgetty
0xd67f2000 00000876 00000867  1  000  stop  0xd67f2270 bash
0xd600e000 00000940 00000876  1  000  run   0xd600e270*lvcreate
kdb> id %eip
0xb800: kdb: Bad user address 0xb800
add    %al,(%eax)
0xb802: add    %al,(%eax)
0xb804: add    %al,(%eax)
0xb806: add    %al,(%eax)
0xb808: add    %al,(%eax)
0xb80a: add    %al,(%eax)
0xb80c: add    %al,(%eax)
0xb80e: add    %al,(%eax)
0xb810: add    %al,(%eax)
0xb812: add    %al,(%eax)
0xb814: add    %al,(%eax)
0xb816: add    %al,(%eax)
0xb818: add    %al,(%eax)
0xb81a: add    %al,(%eax)
0xb81c: add    %al,(%eax)
0xb81e: add    %al,(%eax)
kdb> go
Oops: 0000
CPU:    0
EIP:    0010:[<0000b800>]    Not tainted
EFLAGS: 00010086
eax: ffffffff   ebx: d600e000   ecx: 0000b800   edx: c018fd25
esi: 00000008   edi: d600e000   ebp: d600ff30   esp: d600ff0c
ds: 0018   es: 0018   ss: 0018
Process lvcreate (pid: 940, stackpage=d600f000)
Stack: c011ce83 00000001 00000000 00000008 c01069b9 d600e560 d600ff30 d600e560
       d600ffc4 c01392ff 00000282 d604ab40 d600e000 00000011 c1672144 c011de97
       00000011 d600e568 00000000 d600ff70 d600ffa4 c011e252 00000011 d600ff90
Call Trace: [<c011ce83>] [<c01069b9>] [<c01392ff>] [<c011de97>] [<c011e252>]
   [<c0106d54>]

Code:  Bad EIP value.

- --
Adrian Head

(Public Key available on request.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8OavO8ZJI8OvSkAcRAlktAJ0Uyq3AyRtrrBDkxe5oszt+wrbjAwCfU+wC w9VQ0Hb8tpt8qmCTsM+gJDk=
=5Vw0
-----END PGP SIGNATURE-----

Ok, it looks like that was the problem. XFS wrote the superblock when unfreezing (unlocking) the filesystem, apparently LVM wasn't happy with this I/O on the snapshotted volume before the volume had been created. Something goes quite haywire as a result, but I'm not sure what.

We really don't need the superblock write at this point (it was the last thing done prior to the freeze, no need to do it again), and removing it allows the snapshot to be created without error.

I was curious, though, to see if reversing the order of unlockfs() & lvm_fs_create_lv() at the end of lvm_do_lv_create() might alleviate this problem (i.e. create the lv, THEN unlock the snapshotted volume) but that didn't work, either. Perhaps the LVM folks can comment on this...

Adrian says there's still a problem with overflows; I'll look into that.

-Eric

Eric Sandeen      XFS for Linux     http://oss.sgi.com/projects/xfs
[email protected]   SGI, Inc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I have upgraded the lvm in the kernel to the CVS version from the sistina site. The lvm-1.0.1 upgrade patch it generates is different from the previous one used. The kdb output is pretty much identical though.

What I will do next is to remove ext3 from the kernel and see what happens as well as upgrade to the ext3 CVS and see what happens.

This is the full Oops output when trying to create an XFS snapshot. 2.4.17-xfs
+ lvm-1.0.1 upgrade (from LVM CVS)
+ VFS-lock

Entering kdb (current=0xd5db2000, pid 934) Oops: Oops
due to oops @ 0xc015bc06
eax = 0x748b5356 ebx = 0xc019fdf0 ecx = 0xd7894040 edx = 0x00000000 esi = 0xd5db2000 edi = 0xd7894040 esp = 0xd5db3eac eip = 0xc015bc06 ebp = 0xd7ed4800 xss = 0x00000018 xcs = 0x00000010 eflags = 0x00010286 xds = 0x00000018 xes = 0x00000018 origeax = 0xffffffff &regs = 0xd5db3e78 kdb> bt
EBP EIP Function(args)
0xd7ed4800 0xc015bc06 journal_start+0x36 (0xd7ed4800, 0x1, 0x40018000, 0xd5db3f5c, 0x2)

                               kernel .text 0xc0100000 0xc015bbd0 0xc015bcb0
           0xc0156fd8 ext3_dirty_inode+0x58 (0xd7894040)
                               kernel .text 0xc0100000 0xc0156f80 0xc0157050
           0xc0141b1e __mark_inode_dirty+0x2e (0xd7894040, 0x1)
                               kernel .text 0xc0100000 0xc0141af0 0xc0141b70
           0xc0143001 update_atime+0x51 (0xd7894040, 0x6d6, 0x1, 0x0, 0x6d6)
                               kernel .text 0xc0100000 0xc0142fb0 0xc0143010
           0xc01254ac do_generic_file_read+0x40c (0xd7757840, 0xd7757860, 
0xd5db3f5c, 0xc0125690, 0x6d6)
                               kernel .text 0xc0100000 0xc01250a0 0xc01254c0
           0xc012576a generic_file_read+0x7a (0xd7757840, 0x40018000, 0x1000, 
0xd7757860, 0x0)
                               kernel .text 0xc0100000 0xc01256f0 0xc0125810
           0xc0130cf5 sys_read+0x95 (0x4, 0x40018000, 0x1000, 0x8086ab0, 
0x40018000)
                               kernel .text 0xc0100000 0xc0130c60 0xc0130d30
           0xc0106d1b system_call+0x33
                               kernel .text 0xc0100000 0xc0106ce8 0xc0106d20

kdb> btp 934
EBP EIP Function(args)
0xc019fdf0 0xc015bc06 journal_start+0x36 (0xd7ed4800, 0x1, 0x40018000, 0xd5db3f5c, 0x2)

                               kernel .text 0xc0100000 0xc015bbd0 0xc015bcb0
           0xc0156fd8 ext3_dirty_inode+0x58 (0xd7894040)
                               kernel .text 0xc0100000 0xc0156f80 0xc0157050
           0xc0141b1e __mark_inode_dirty+0x2e (0xd7894040, 0x1)
                               kernel .text 0xc0100000 0xc0141af0 0xc0141b70
           0xc0143001 update_atime+0x51 (0xd7894040, 0x6d6, 0x1, 0x0, 0x6d6)
                               kernel .text 0xc0100000 0xc0142fb0 0xc0143010
           0xc01254ac do_generic_file_read+0x40c (0xd7757840, 0xd7757860, 
0xd5db3f5c, 0xc0125690, 0x6d6)
                               kernel .text 0xc0100000 0xc01250a0 0xc01254c0
           0xc012576a generic_file_read+0x7a (0xd7757840, 0x40018000, 0x1000, 
0xd7757860, 0x0)
                               kernel .text 0xc0100000 0xc01256f0 0xc0125810
           0xc0130cf5 sys_read+0x95 (0x4, 0x40018000, 0x1000, 0x8086ab0, 
0x40018000)
                               kernel .text 0xc0100000 0xc0130c60 0xc0130d30
           0xc0106d1b system_call+0x33
                               kernel .text 0xc0100000 0xc0106ce8 0xc0106d20
kdb> ps
Task Addr  Pid      Parent   [*] cpu  State Thread     Command
0xd7fe2000 00000001 00000000  1  000  stop  0xd7fe2270 init
0xc163c000 00000002 00000001  1  000  stop  0xc163c270 keventd
0xc1638000 00000003 00000000  1  000  stop  0xc1638270 ksoftirqd_CPU0
0xc1636000 00000004 00000000  1  000  stop  0xc1636270 kswapd
0xc1634000 00000005 00000000  1  000  stop  0xc1634270 bdflush
0xc1632000 00000006 00000000  1  000  stop  0xc1632270 kupdated
0xd7ec4000 00000007 00000001  1  000  stop  0xd7ec4270 mdrecoveryd
0xd7eae000 00000008 00000001  1  000  stop  0xd7eae270 raid5d
0xd7b32000 00000009 00000001  1  000  stop  0xd7b32270 kjournald
0xd72a6000 00000153 00000001  1  000  stop  0xd72a6270 kjournald
0xd729a000 00000154 00000001  1  000  stop  0xd729a270 kjournald
0xd7292000 00000155 00000001  1  000  stop  0xd7292270 kjournald
0xd7172000 00000157 00000001  1  000  stop  0xd7172270 kreiserfsd
0xd70ac000 00000158 00000001  1  000  stop  0xd70ac270 kjournald
0xd6f68000 00000160 00000001  1  000  stop  0xd6f68270 pagebuf_daemon
0xd6256000 00000523 00000001  1  000  stop  0xd6256270 dhcpcd
0xd77e0000 00000659 00000001  1  000  stop  0xd77e0270 syslogd
0xd6212000 00000664 00000001  1  000  stop  0xd6212270 klogd
0xd61a2000 00000684 00000001  1  000  stop  0xd61a2270 portmap
0xd60aa000 00000712 00000001  1  000  stop  0xd60aa270 rpc.statd
0xd5f74000 00000825 00000001  1  000  stop  0xd5f74270 crond
more>
0xd5f5e000 00000861 00000001  1  000  stop  0xd5f5e270 atd
0xd602c000 00000868 00000001  1  000  stop  0xd602c270 login
0xd5f48000 00000869 00000001  1  000  stop  0xd5f48270 mingetty
0xd5fa2000 00000870 00000001  1  000  stop  0xd5fa2270 mingetty
0xd5f5a000 00000871 00000001  1  000  stop  0xd5f5a270 mingetty
0xd63a8000 00000872 00000001  1  000  stop  0xd63a8270 mingetty
0xd6008000 00000873 00000001  1  000  stop  0xd6008270 mingetty
0xd5f76000 00000874 00000001  1  000  stop  0xd5f76270 mgetty
0xd74aa000 00000877 00000868  1  000  stop  0xd74aa270 bash
0xd5db2000 00000934 00000877  1  000  run   0xd5db2270*lvcreate
kdb> id %eip
0xc015bc06 journal_start+0x36:  cmp    %ebp,(%eax)
0xc015bc08 journal_start+0x38:  je     0xc015bc2d journal_start+0x5d:
0xc015bc0a journal_start+0x3a:  push   $0xc022e920
0xc015bc0f journal_start+0x3f:  push   $0xe1
0xc015bc14 journal_start+0x44:  push   $0xc022a62b
0xc015bc19 journal_start+0x49:  push   $0xc0229f00
0xc015bc1e journal_start+0x4e:  push   $0xc022c9e0
0xc015bc23 journal_start+0x53:  call   0xc0115380 printk:
0xc015bc28 journal_start+0x58:  ud2a
0xc015bc2a journal_start+0x5a:  add    $0x14,%esp
0xc015bc2d journal_start+0x5d:  incl   0x8(%ebx)
0xc015bc30 journal_start+0x60:  jmp    0xc015bca0 journal_start+0xd0:
0xc015bc32 journal_start+0x62:  push   $0x1
0xc015bc34 journal_start+0x64:  push   $0xf0
0xc015bc39 journal_start+0x69:  push   $0x14
0xc015bc3b journal_start+0x6b:  push   $0xc0229f00
kdb> go
Oops: 0000
CPU:    0
EIP:    0010:[<c015bc06>]    Not tainted
EFLAGS: 00010286
eax: 748b5356   ebx: c019fdf0   ecx: d7894040   edx: 00000000
esi: d5db2000   edi: d7894040   ebp: d7ed4800   esp: d5db3eac
ds: 0018   es: 0018   ss: 0018
Process lvcreate (pid: 934, stackpage=d5db3000)
Stack: c019fdf0 c019fdf0 ffffffe2 d7894040 00000000 c0156fd8 d7ed4800 00000001
       40018000 d5db3f5c 00000002 00000018 d7894040 d7b41000 00000001 c0141b1e
       d7894040 3c385279 d7757860 d78940f0 c0143001 d7894040 00000001 00000000
Call Trace: [<c019fdf0>] [<c019fdf0>] [<c0156fd8>] [<c0141b1e>] [<c0143001>]
   [<c01254ac>] [<c012576a>] [<c0125690>] [<c0130cf5>] [<c0106d1b>]

Code: 39 28 74 23 68 20 e9 22 c0 68 e1 00 00 00 68 2b a6 22 c0 68

- --
Adrian Head

(Public Key available on request.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8N+0o8ZJI8OvSkAcRAgk+AJ4/0P4qg1A2UnABK5O+0kM9jYlYfgCgn+fv GRJHFM6+Mt9CJNg9n13XoEI=
=ZVhu
-----END PGP SIGNATURE-----

This patch, on top of xfs cvs (2.4.17-xfs), with the LVM 1.0.1 upgrade patch applied, then the VFS lock patch, works for me. Note that this is stolen from the 1.0.1rc4(ish) version of LVM in Linus' kernel. It uses the built-in blocks from lv_snap->lv_iobuf->blocks rather than declaring a new chunk of blocks.

I tested snapshot creation & overflow with ext2, ext3, reiser, and xfs. I have not looked closely enough to see why xfs breaks w/o this patch, and why it works with it, but... give it a shot.

-Eric

Eric Sandeen      XFS for Linux     http://oss.sgi.com/projects/xfs
[email protected]   SGI, Inc.

@@ -349,7 +349,6 @@
 	unsigned long phys_start;
 	int idx = lv_snap->lv_remap_ptr, chunk_size = lv_snap->lv_chunk_size;
 	struct kiobuf * iobuf;
-	unsigned long blocks[KIO_MAX_SECTORS];
 	int blksize_snap, blksize_org, min_blksize, max_blksize;
 	int max_sectors, nr_sectors;
 

@@ -400,19 +399,19 @@
 
 		iobuf->length = nr_sectors << 9;
 
-		if (!lvm_snapshot_prepare_blocks(blocks, phys_start,
+		if (!lvm_snapshot_prepare_blocks(iobuf->blocks, phys_start,
 						 nr_sectors, blksize_org))
 			goto fail_prepare;
 
-		if (__brw_kiovec(READ, 1, &iobuf, org_phys_dev, blocks,
+		if (__brw_kiovec(READ, 1, &iobuf, org_phys_dev, iobuf->blocks,
 				 blksize_org, lv_snap) != (nr_sectors<<9))
 			goto fail_raw_read;
 
-		if (!lvm_snapshot_prepare_blocks(blocks, snap_start,
+		if (!lvm_snapshot_prepare_blocks(iobuf->blocks, snap_start,
 						 nr_sectors, blksize_snap))
 			goto fail_prepare;
 
-		if (__brw_kiovec(WRITE, 1, &iobuf, snap_phys_dev, blocks,
+		if (__brw_kiovec(WRITE, 1, &iobuf, snap_phys_dev, iobuf->blocks,
 				 blksize_snap, lv_snap) != (nr_sectors<<9))
 			goto fail_raw_write;
 

Ah, ok. Steve pointed out that in the stock LVM version,

unsigned long blocks[KIO_MAX_SECTORS];

plops 4k down on the stack; xfs's stack must be a bit bigger than other filesystems, and it overflows... that's why xfs was failing with that code, and other filesystems survived.

-Eric

Eric Sandeen      XFS for Linux     http://oss.sgi.com/projects/xfs
[email protected]   SGI, Inc.