Skip to contentSkip to conversation

New CentOS4/RHEL4-compatible xfs module rpms

DISCUSSION Eric Sandeen 5 participants 25 messages Nov 19, 2006 – Jul 21, 2007

lore
Share & tools
lore
#0 lore raw

http://sandeen.net/rhel4_xfs/kernel-module-xfs-2.6.9-42.0.2.EL-0.2-1.src.rpm

rpmbuild --rebuild --target i686 kernel-module-xfs-2.6.9-42.0.2.EL-0.2-1.src.rpm

will build against the currently running kernel, or

rpmbuild --rebuild --target i686 --define "kernel_topdir /lib/modules/2.6.9-42.0.2.EL/build" kernel-module-xfs-2.6.9-42.0.2.EL-0.2-1.src.rpm

will build against what is defined in kernel_topdir

you need matching kernel & kernel-devel rpms installed to build.

Changelog: mostly pulling in fixes sgi sent for sles9, plus specfile cleanups.

[root@sandeen rhel4_xfs]# rpm -qp --changelog kernel-module-xfs-2.6.9-42.0.2.EL-0.2-1.src.rpm
* Mon Nov 13 2006 - [email protected]

- removed xfs_direct_io_locking.patch, RHEL4U4 has the fix now.
- Update to xfs codebase from SLES9_SP3_BRANCH_20061107171129
- xfs-kern-26347a-fix-race-on-link: fix race on link (191713, SGI:PV953287).
- xfs-kern-26040a-do-not-dirty-inode-being-freed:
Don't dirty the inode if it being freed in xfs_iunpin (179117, SGI:PV952967).
- xfs-kern-25687a-sles9sp3-iunpin-reclaim-fix: Fix
an inode use-after-free during an unpin (SGI:PV946321, 142533).
- xfs-kern-930841-default_acl_enospc_fix:
Default acl ENOSPC fix (133990, SGI:PV930841).
- xfs-kern-25238a-quota-trans-diag: quota trans diag (131262, SGI:PV931456).
- xfs-ftruncate-stale-data: XFS ftruncate() bug
could expose stale data (151055).
- xfs-log-diag: log_runout_diagnostics (131262, SGI:PV947110).
- xfs-kern-202363a-fix-xfs_finish_reclaim_all-umount-deadlock:
fix xfs_finish_reclaim_all umount deadlock (132358, SGI:PV943821).
- clean up specfile:
default to building against running kernel
try to match fedora conventions slightly better re-enable debuginfo pkg building, module stripping fix recognition of root on xfs, (grep -i XFS)
remove unused -source subpackage

-Eric

#1 lore raw
Quoted 5 lines · Eric Sandeen

http://sandeen.net/rhel4_xfs/kernel-module-xfs-2.6.9-42.0.2.EL-0.2-1.src.rpm

rpmbuild --rebuild --target i686 kernel-module-xfs-2.6.9-42.0.2.EL-0.2-1.src.rpm

Quoted 8 lines

will build against the currently running kernel, or

rpmbuild --rebuild --target i686 --define "kernel_topdir /lib/modules/2.6.9-42.0.2.EL/build" kernel-module-xfs-2.6.9-42.0.2.EL-0.2-1.src.rpm

will build against what is defined in kernel_topdir

you need matching kernel & kernel-devel rpms installed to build.

I built this for RHELAS4U2 x86_64 with the 2.6.9-22.ELsmp kernel.

I did enable the xfs_direct_io_locking.patch (see inside kernel-module-xfs.spec for details).

However, I run into issues with xfs_freeze as it often locks up when I try to freeze a file system where there is I/O activity. Sometimes it happen on the first xfs_freeze invocation to freeze the file system, other times I have to unfreeze and then it happens on the second time I freeze. xfs_freeze never returns when this happens.

Looks like xfs_io get stuck --see partial output from ps auxf:

strace -ff -o freeze.txt xfs_freeze -f /mnt/xfs
\_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs

Anyone else encountering this issue?

Signature · 1 line
Geir A. Myrestrand
#2 lore raw
Quoted 6 lines · Geir A. Myrestrand

However, I run into issues with xfs_freeze as it often locks up when I try to freeze a file system where there is I/O activity. Sometimes it happen on the first xfs_freeze invocation to freeze the file system, other times I have to unfreeze and then it happens on the second time I freeze. xfs_freeze never returns when this happens.

Quoted 8 lines

Looks like xfs_io get stuck --see partial output from ps auxf:

strace -ff -o freeze.txt xfs_freeze -f /mnt/xfs
\_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs

Anyone else encountering this issue?

Did you have this problem on the previous version?

You might do sysrq-t (echo t > /proc/sysrq-trigger) and see where the thread is stuck.

-Eric

#3 lore raw
Quoted 10 lines · Eric Sandeen
Quoted 8 lines · Geir A. Myrestrand

However, I run into issues with xfs_freeze as it often locks up when I try to freeze a file system where there is I/O activity. Sometimes it happen on the first xfs_freeze invocation to freeze the file system, other times I have to unfreeze and then it happens on the second time I freeze. xfs_freeze never returns when this happens.

Looks like xfs_io get stuck --see partial output from ps auxf:

strace -ff -o freeze.txt xfs_freeze -f /mnt/xfs
\_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs

Quoted 7 lines
Quoted 2 lines

Anyone else encountering this issue?

Did you have this problem on the previous version?

You might do sysrq-t (echo t > /proc/sysrq-trigger) and see where the thread is stuck.

I reproduced it and dumped the task list.

The user-mode XFS processes were:

root      2816  0.0  0.1  2812  628 pts/3    S+   12:58   0:00  | \_ strace -ff -o freeze.txt xfs_freeze -f /mnt/xfs
root      2847  0.0  0.2 52752 1048 pts/3    T+   12:58   0:00  | 
    \_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs

root 2891 0.0 0.1 49976 600 pts/3 D+ 12:58 0:00 |
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs

My I/O generating processes were:

root      3832  1.9  0.2 59644 1376 pts/1    S+   12:57   0:17  | \_ /usr/bin/perl ./test.pl
root      2992  0.0  0.0 49912  404 pts/1    D+   12:58   0:00  | 
    \_ touch 857

Here are some relevant information from /var/log/messages (I stuck in a divider when I pasted in sections that weren't just right after the previous section):

pdflush D ffffffff8014a190 0 48 6 50 47 (L-TLB)
000001001f99dcb8 0000000000000046 0000000080134722 ffffffffa0054a51

       000000001f99dc30 0000000000000000 000001001fa2e310 0000000000000000
       000001001f9737f0 00000000000000d9
Call Trace:<ffffffffa0054a51>{:ext3:ext3_ordered_writepage+0}
       <ffffffffa01d96b5>{:xfs:xlog_state_sync_all+456} 
<ffffffffa01f4b9c>{:xfs:pagebuf_rele+54}
       <ffffffff8014a190>{keventd_create_kthread+0} 
<ffffffff80302637>{__down+147}
       <ffffffff80132e8d>{default_wake_function+0} 
<ffffffff80303c13>{__down_failed+53}
       <ffffffffa01e59da>{:xfs:xfs_sync+0} 
<ffffffffa01f64ad>{:xfs:.text.lock.xfs_buf+15}
       <ffffffffa01df1fa>{:xfs:xfs_getsb+37} 
<ffffffffa01e5817>{:xfs:xfs_syncsub+2507}
       <ffffffff8014a190>{keventd_create_kthread+0} 
<ffffffff8010f459>{__down_trylock+68}
       <ffffffff8014a190>{keventd_create_kthread+0} 
<ffffffffa01f347a>{:xfs:linvfs_write_super+33}
       <ffffffff8017ca9e>{sync_supers+167} 
<ffffffff8015d8e8>{wb_kupdate+36}
       <ffffffff8015e38c>{pdflush+323} <ffffffff8015d8c4>{wb_kupdate+0}
       <ffffffff8015e249>{pdflush+0} <ffffffff8014a167>{kthread+200}
       <ffffffff80110ca3>{child_rip+8} 
<ffffffff8014a190>{keventd_create_kthread+0}
       <ffffffff8014a09f>{kthread+0} <ffffffff80110c9b>{child_rip+0}

----------------------------------------------------------------------------

xfs S 0000000000000006 0 2723 1 2742 2697 (NOTLB)
00000100156e9d78 0000000000000002 00000100156e9d98 0000010000000074

       000001001eb87880 000001001eb878f0 000000d000000000 0000000100000246
       00000100154cc030 0000000000005f88
Call Trace:<ffffffff8013ea74>{__mod_timer+293} <ffffffff803039b3>{schedule_timeout+244}
       <ffffffff8013f49e>{process_timeout+0} 
<ffffffff8018902b>{do_select+939}
       <ffffffff80188bc5>{__pollwait+0} <ffffffff801893aa>{sys_select+820}
       <ffffffff801908fc>{dnotify_parent+34} 

<ffffffff80110052>{system_call+126}

----------------------------------------------------------------------------

xfslogd/0 S ffffffffa01f54b0 0 3818 6 3819 1981 (L-TLB)
00000100104e5e68 0000000000000046 0000000000000283 0000010011a268e0

       ffffffffa01f54b0 0000000000000246 ffffffff80303cc1 00000000a01bb7a8
       0000010011d377f0 000000000000023e
Call Trace:<ffffffffa01f54b0>{:xfs:pagebuf_iodone_work+0} <ffffffff80303cc1>{__up_wakeup+53}
       <ffffffffa01f54b0>{:xfs:pagebuf_iodone_work+0} 
<ffffffff80146431>{worker_thread+226}
       <ffffffff80132e8d>{default_wake_function+0} 
<ffffffff80132e8d>{default_wake_function+0}
       <ffffffff8014a190>{keventd_create_kthread+0} 
<ffffffff8014634f>{worker_thread+0}
       <ffffffff8014a190>{keventd_create_kthread+0} 
<ffffffff8014a167>{kthread+200}
       <ffffffff80110ca3>{child_rip+8} 
<ffffffff8014a190>{keventd_create_kthread+0}
       <ffffffff8014a09f>{kthread+0} <ffffffff80110c9b>{child_rip+0}

xfslogd/1 S ffffffffa01f54b0 0 3819 6 3820 3818 (L-TLB)
00000100105d7e68 0000000000000046 0000000000000287 ffffffffa01e2960

       000001000e48eb20 000001000e48eb20 0000010012d7e260 0000000100000246
       000001001053f7f0 000000000000022f
Call Trace:<ffffffffa01e2960>{:xfs:xfs_trans_delete_ail+45} <ffffffffa01f54b0>{:xfs:pagebuf_iodone_work+0}
       <ffffffff80146431>{worker_thread+226} 
<ffffffff80132e8d>{default_wake_function+0}
       <ffffffff80132e8d>{default_wake_function+0} 
<ffffffff8014a190>{keventd_create_kthread+0}
       <ffffffff8014634f>{worker_thread+0} 
<ffffffff8014a190>{keventd_create_kthread+0}
       <ffffffff8014a167>{kthread+200} <ffffffff80110ca3>{child_rip+8}
       <ffffffff8014a190>{keventd_create_kthread+0} 
<ffffffff8014a09f>{kthread+0}
       <ffffffff80110c9b>{child_rip+0}
xfsdatad/0    S 00000100164aa9c0     0  3820      6          3821  3819 (L-TLB)
00000100104e1e68 0000000000000046 000000000032a7bd 000000190000006a
       000001001053e7f0 000000000000006a 0000010001707840 0000000000187397
       000001001053f030 0000000000000d8a
Call Trace:<ffffffff8014a190>{keventd_create_kthread+0} <ffffffff8014634f>{worker_thread+0}
       <ffffffff80146431>{worker_thread+226} 
<ffffffff80132e8d>{default_wake_function+0}
       <ffffffff80132e8d>{default_wake_function+0} 
<ffffffff8014a190>{keventd_create_kthread+0}
       <ffffffff8014634f>{worker_thread+0} 
<ffffffff8014a190>{keventd_create_kthread+0}
       <ffffffff8014a167>{kthread+200} <ffffffff80110ca3>{child_rip+8}
       <ffffffff8014a190>{keventd_create_kthread+0} 
<ffffffff8014a09f>{kthread+0}
       <ffffffff80110c9b>{child_rip+0}
xfsdatad/1    S 00000100164aaa40     0  3821      6                3820 (L-TLB)
00000100104e3e68 0000000000000046 000000260000000a 0000001900000074
       000001001f9bd030 0000000000000074 000001000170f840 000000010032ba4b
       000001001053e7f0 0000000000000b24
Call Trace:<ffffffff8014a190>{keventd_create_kthread+0} <ffffffff8014634f>{worker_thread+0}
       <ffffffff80146431>{worker_thread+226} 
<ffffffff80132e8d>{default_wake_function+0}
       <ffffffff80132e8d>{default_wake_function+0} 
<ffffffff8014a190>{keventd_create_kthread+0}
       <ffffffff8014634f>{worker_thread+0} 
<ffffffff8014a190>{keventd_create_kthread+0}
       <ffffffff8014a167>{kthread+200} <ffffffff80110ca3>{child_rip+8}
       <ffffffff8014a190>{keventd_create_kthread+0} 
<ffffffff8014a09f>{kthread+0}
       <ffffffff80110c9b>{child_rip+0}
xfsbufd       S 000000010027d0c0     0  3822      1          3827  3815 (L-TLB)
00000100111fdea8 0000000000000046 ffffffff803d9920 0000001900000073
       000001001f9bd030 0000000000000073 0000010001707840 0000000080138348
       000001001053e030 000000000000019b
Call Trace:<ffffffff8013ea74>{__mod_timer+293} <ffffffff803039b3>{schedule_timeout+244}
       <ffffffff8013f49e>{process_timeout+0} 
<ffffffffa01f62db>{:xfs:xfsbufd+172}
       <ffffffff80110ca3>{child_rip+8} 
<ffffffff8011de14>{flat_send_IPI_mask+0}
       <ffffffffa01f622f>{:xfs:xfsbufd+0} <ffffffff80110c9b>{child_rip+0}

xfssyncd S ffffffffa01f36b9 0 3827 1 3822 (L-TLB)
0000010011f0fea8 0000000000000046 ffffffffa01f36b9 0000000000000202

       ffffffff8010f459 00000100126e2d40 000001001d5e3980 0000000100000000
       000001001e4dd7f0 00000000000025eb
Call Trace:<ffffffffa01f36b9>{:xfs:linvfs_fill_super+0} <ffffffff8010f459>{__down_trylock+68}
       <ffffffff8013ea74>{__mod_timer+293} 
<ffffffffa01f36b9>{:xfs:linvfs_fill_super+0}
       <ffffffff803039b3>{schedule_timeout+244} 
<ffffffff8013f49e>{process_timeout+0}
       <ffffffffa01f323c>{:xfs:xfssyncd+120} 
<ffffffff80110ca3>{child_rip+8}
       <ffffffffa01f36b9>{:xfs:linvfs_fill_super+0} 
<ffffffff801cc053>{dummy_d_instantiate+0}
       <ffffffffa01f31c4>{:xfs:xfssyncd+0} <ffffffff80110c9b>{child_rip+0}

----------------------------------------------------------------------------

xfs_freeze t 00000000006c51e0 0 2847 2816 2891 (NOTLB)
00000100074d7e78 0000000000000002 0000010015654030 00000100111fec40

       0000000000000206 ffffffff801410a6 0000000000000011 0000000000040004
       0000010015654030 0000000000001a97
Call Trace:<ffffffff801410a6>{do_notify_parent_cldstop+243} <ffffffff801418d8>{ptrace_stop+315}
       <ffffffff801419c4>{ptrace_notify+139} 
<ffffffff8011419b>{syscall_trace+37}
       <ffffffff8011420f>{syscall_trace_enter+43} 

<ffffffff80110156>{tracesys+113}

xfs_io D 00000100146af380 0 2891 2816 2847 (NOTLB)
00000100075c9af8 0000000000000002 000001000a415b80 0000000000000064

       000000000007d000 ffffffff802499ca ffffffff80134722 00000001075c9a90
       000001000e83f7f0 0000000001a7ca67
Call Trace:<ffffffff802499ca>{generic_make_request+355} <ffffffff80134722>{autoremove_wake_function+0}
       <ffffffff80302637>{__down+147} 
<ffffffff80132e8d>{default_wake_function+0}
       <ffffffff80303c13>{__down_failed+53} 
<ffffffffa01f2a45>{:xfs:xfs_bdstrat_cb+0}
       <ffffffffa01f64ad>{:xfs:.text.lock.xfs_buf+15} 
<ffffffffa01f60f6>{:xfs:xfs_flush_buftarg+213}
       <ffffffffa01e59af>{:xfs:xfs_quiesce_fs+69} 
<ffffffffa01f34c1>{:xfs:linvfs_sync_super+65}
       <ffffffff80178367>{freeze_bdev+215} 
<ffffffffa01f0cd1>{:xfs:xfs_ioctl+5161}
       <ffffffff801588f0>{find_get_page+65} 
<ffffffff801593d2>{filemap_nopage+378}
       <ffffffff80131c6d>{finish_task_switch+55} 
<ffffffff80303358>{thread_return+42}
       <ffffffff8014191f>{ptrace_stop+386} 
<ffffffff801419e0>{ptrace_notify+167}
       <ffffffffa01ef380>{:xfs:linvfs_ioctl+112} 
<ffffffff80188631>{sys_ioctl+853}
       <ffffffff801101b6>{tracesys+209}
touch         D 0000000000239000     0  2992   3832 (NOTLB)
0000010008dd76e8 0000000000000006 0000000000000001 0000000000000001
       0000000000000016 ffffffff80131931 0000000100000000 0000000000000003
       000001001ed54030 00000000000078de
Call Trace:<ffffffff80131931>{try_to_wake_up+734} <ffffffffa01f4dc2>{:xfs:_pagebuf_find+327}
       <ffffffff80302637>{__down+147} 
<ffffffff80132e8d>{default_wake_function+0}
       <ffffffff80303c13>{__down_failed+53} 
<ffffffffa01f64ad>{:xfs:.text.lock.xfs_buf+15}
       <ffffffffa01f4de1>{:xfs:_pagebuf_find+358} 
<ffffffffa01f5065>{:xfs:xfs_buf_get_flags+100}
       <ffffffffa01b1be0>{:xfs:xfs_bmap_search_extents+92}
       <ffffffffa01f5b01>{:xfs:xfs_buf_read_flags+16} 
<ffffffffa01e30b3>{:xfs:xfs_trans_read_buf+428}
       <ffffffffa01bc37c>{:xfs:xfs_da_do_buf+984} 
<ffffffffa01bc605>{:xfs:xfs_da_read_buf+21}
       <ffffffffa01bc87b>{:xfs:xfs_da_node_lookup_int+145}
       <ffffffffa01bc87b>{:xfs:xfs_da_node_lookup_int+145}
       <ffffffffa01c58a2>{:xfs:xfs_dir2_node_addname+83} 
<ffffffffa01d26bc>{:xfs:xfs_ichgtime+93}
       <ffffffff8018f975>{wake_up_inode+6} 
<ffffffffa01b286b>{:xfs:xfs_bmap_last_offset+179}
       <ffffffffa01c056e>{:xfs:xfs_dir2_createname+283} 
<ffffffffa01e8818>{:xfs:xfs_create+992}
       <ffffffff801cbecc>{dummy_inode_permission+0} 
<ffffffffa01f0fb7>{:xfs:linvfs_mknod+453}
       <ffffffffa01bbf7c>{:xfs:xfs_da_brelse+116} 
<ffffffffa01c6167>{:xfs:xfs_dir2_node_lookup+170}
       <ffffffffa01c042d>{:xfs:xfs_dir2_lookup+248} 
<ffffffff80184d4b>{vfs_create+214}
       <ffffffff80185179>{open_namei+430} <ffffffff80176558>{filp_open+39}
       <ffffffff801ea045>{strncpy_from_user+74} 
<ffffffff80176661>{get_unused_fd+230}
       <ffffffff80176750>{sys_open+57} <ffffffff80110052>{system_call+126}
Signature · 1 line
Geir A. Myrestrand
#4 lore raw
Quoted 9 lines · Geir A. Myrestrand
Quoted 8 lines · Eric Sandeen
Quoted 6 lines · Geir A. Myrestrand

However, I run into issues with xfs_freeze as it often locks up when I try to freeze a file system where there is I/O activity. Sometimes it happen on the first xfs_freeze invocation to freeze the file system, other times I have to unfreeze and then it happens on the second time I freeze. xfs_freeze never returns when this happens.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Looks like xfs_io get stuck --see partial output from ps auxf:

strace -ff -o freeze.txt xfs_freeze -f /mnt/xfs
\_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs

Anyone else encountering this issue?

Yes, and I fixed it about a 2 weeks ago. It's an ABBA deadlock between lookup of multiple, already dirty, metadata buffers and synchronous buftarg flushing (that occurs when trying to freeze a filesystem)

<sigh>

I just went looking for the Take message in the archive, and it is not there. I cc all my takes to [email protected], so I'm not sure why it isn't in the archive....

http://oss.sgi.com/archives/xfs/2006-11/msg00291.html

Was a followup cleanup of a problem found during review of the fix for the freeze problem.

The text of the take message fo rthe fix is:

Fix a synchronous buftarg flush deadlock when freezing.

At the last stage of a freeze, we flush the buftarg synchronously over and over again until it succeeds twice without skipping any buffers.

The delwri list flush skips pinned buffers, but tries to flush all others. It removes the buffers from the delwri list, then tries to lock them one at a time as it traverses the list to issue the I/O. It holds them locked until we issue all of the I/O and then unlocks them once we've waited for it to complete.

The problem is that during a freeze, the filesystem may still be doing stuff - like flushing delalloc data buffers - in the background and hence we can be trying to lock buffers that were on the delwri list at the same time. Hence we can get ABBA deadlocks between threads doing allocation and the buftarg flush (freeze) thread.

Fix it by skipping locked (and pinned) buffers as we traverse the delwri buffer list.

----

And the diff was:

http://oss.sgi.com/cgi-bin/cvsweb.cgi/linux-2.6-xfs/fs/xfs/linux-2.6/xfs_buf.c.diff?r1=1.229;r2=1.230

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
#5 lore raw
Quoted 11 lines · David Chinner
Quoted 10 lines · Geir A. Myrestrand
Quoted 10 lines
Quoted 8 lines · Geir A. Myrestrand

However, I run into issues with xfs_freeze as it often locks up when I try to freeze a file system where there is I/O activity. Sometimes it happen on the first xfs_freeze invocation to freeze the file system, other times I have to unfreeze and then it happens on the second time I freeze. xfs_freeze never returns when this happens.

Looks like xfs_io get stuck --see partial output from ps auxf:

strace -ff -o freeze.txt xfs_freeze -f /mnt/xfs
\_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs

Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

Anyone else encountering this issue?

Yes, and I fixed it about a 2 weeks ago. It's an ABBA deadlock between lookup of multiple, already dirty, metadata buffers and synchronous buftarg flushing (that occurs when trying to freeze a filesystem)

<sigh>

That is awesome news, mate! :-)

Quoted 6 lines

The problem is that during a freeze, the filesystem may still be doing stuff - like flushing delalloc data buffers - in the background and hence we can be trying to lock buffers that were on the delwri list at the same time. Hence we can get ABBA deadlocks between threads doing allocation and the buftarg flush (freeze) thread.

That sounds like an accurate description of my test environment. I bet this is the issue...

Quoted 2 lines

Fix it by skipping locked (and pinned) buffers as we traverse the delwri buffer list.

Good to know that you're one step ahead!

Quoted 3 lines

And the diff was:

http://oss.sgi.com/cgi-bin/cvsweb.cgi/linux-2.6-xfs/fs/xfs/linux-2.6/xfs_buf.c.diff?r1=1.229;r2=1.230

Excellent. I will try this tomorrow (it's late in the evening here in New York now). I'll let you know how it works out.

Thanks!

Signature · 1 line
Geir A. Myrestrand

begin:vcard
fn:Geir A. Myrestrand
n:Myrestrand;Geir A.
org:FalconStor Software, Inc.
adr:Suite 2S01;;2 Huntington Quadrangle;Melville;NY;11747;U.S.A. email;internet:[email protected] title:Senior Software Engineer
tel;work:(631) 773-5842
tel;cell:(631) 747-5049
note;quoted-printable:Skype: gmyrestrand=0D=0A=
MSN Messenger: [email protected]=0D=0A=

x-mozilla-html:FALSE
url:http://www.falconstor.com
version:2.1
end:vcard

#6 lore raw
In reply to Eric Sandeen
Quoted 1 line · Eric Sandeen

Did you have this problem on the previous version?

Not sure if this occur with the older XFS modules, because I haven't tested it.

Signature · 1 line
Geir A. Myrestrand
#7 lore raw
In reply to Eric Sandeen
Quoted 6 lines · Eric Sandeen
Quoted 2 lines · Geir A. Myrestrand

However, I run into issues with xfs_freeze as it often locks up when I try to freeze a file system where there is I/O activity. Sometimes it happen on the first xfs_freeze invocation to freeze the file system, other times I have to unfreeze and then it happens on the second time I freeze. xfs_freeze never returns when this happens.

Did you have this problem on the previous version?

I just tested with the old version, and I run into the same issue.

Signature · 1 line
Geir A. Myrestrand
#8 lore raw
Quoted 10 lines · Geir A. Myrestrand
Quoted 9 lines · Eric Sandeen
Quoted 6 lines · Geir A. Myrestrand

However, I run into issues with xfs_freeze as it often locks up when I try to freeze a file system where there is I/O activity. Sometimes it happen on the first xfs_freeze invocation to freeze the file system, other times I have to unfreeze and then it happens on the second time I freeze. xfs_freeze never returns when this happens.

Quoted 4 lines
Quoted 1 line

Did you have this problem on the previous version?

I just tested with the old version, and I run into the same issue.

Does it happen with mainline (current) kernels? This is ringing some distant bells, something to do with an atomic_inc being in the wrong spot in [_]xfs_trans_alloc ... or maybe a wrong call to xfs_trans_alloc instead of the '_' prefixed version - details are murky now though. I do remember fixing ... something in here, a few months back, that fix may not be in Eric's codebase yet.

cheers.

Signature · 1 line
Nathan
#9 lore raw
Quoted 8 lines · Nathan Scott

Does it happen with mainline (current) kernels? This is ringing some distant bells, something to do with an atomic_inc being in the wrong spot in [_]xfs_trans_alloc ... or maybe a wrong call to xfs_trans_alloc instead of the '_' prefixed version - details are murky now though. I do remember fixing ... something in here, a few months back, that fix may not be in Eric's codebase yet.

cheers.

I've only tried this on RHELAS4 with the kernel shipped with Update 2. I have not tried to update the kernel, or install Update 4 or whatever is current now. My kernel is 2.6.9-22.ELsmp.

It wouldn't be easy for me to switch to a newer kernel, because it is not just a matter of my machine --we have a product built for this particular configuration. Switching to a new kernel would reset our QA efforts.

If this may be fixed in the XFS modules on the other hand, then it would be awesome if we could get it resolved with an updated XFS package for RHEL4. I would be more than willing to put in an extra effort and work with anyone on this.

Signature · 1 line
Geir A. Myrestrand
#10 lore raw
Quoted 5 lines · Geir A. Myrestrand

It wouldn't be easy for me to switch to a newer kernel, because it is not just a matter of my machine --we have a product built for this particular configuration. Switching to a new kernel would reset our QA efforts.

You misunderstood me I think - I didn't suggest switching kernels, just that you test out the latest. If its OK, then its relatively easy to search for a change that fixed it. If its not OK, then theres a bug in mainline which should get some attention too.

cheers.

Signature · 1 line
Nathan
#11 lore raw
Quoted 7 lines · Nathan Scott
Quoted 1 line · Geir A. Myrestrand

It wouldn't be easy for me to switch to a newer kernel, because it is not just a matter of my machine --we have a product built for this particular configuration. Switching to a new kernel would reset our QA efforts.

You misunderstood me I think - I didn't suggest switching kernels, just that you test out the latest. If its OK, then its relatively easy to search for a change that fixed it. If its not OK, then theres a bug in mainline which should get some attention too.

Sorry Nathan, but I'm not sure I understand what you refer to with "test out the latest" --are you referring to the kernel or XFS?

Signature · 1 line
Geir A. Myrestrand
#12 lore raw
Quoted 2 lines · Geir A. Myrestrand

Sorry Nathan, but I'm not sure I understand what you refer to with "test out the latest" --are you referring to the kernel or XFS?

Yes. :) Current mainline kernels, or CVS from oss.sgi.com would have this fix I'm thinking of.

cheers.

Signature · 1 line
Nathan
#13 lore raw
Quoted 6 lines · Nathan Scott
Quoted 2 lines · Geir A. Myrestrand

Sorry Nathan, but I'm not sure I understand what you refer to with "test out the latest" --are you referring to the kernel or XFS?

Yes. :) Current mainline kernels, or CVS from oss.sgi.com would have this fix I'm thinking of.

Thanks for the "clarification"... ;-)

I'll try to update the XFS code first, hopefully it won't break too many of the patches Eric had in his source RPM...but I will try to correct them if they fail to apply cleanly...

I'll keep you posted.

Signature · 1 line
Geir A. Myrestrand

begin:vcard
fn:Geir A. Myrestrand
n:Myrestrand;Geir A.
org:FalconStor Software, Inc.
adr:Suite 2S01;;2 Huntington Quadrangle;Melville;NY;11747;U.S.A. email;internet:[email protected] title:Senior Software Engineer
tel;work:(631) 773-5842
tel;cell:(631) 747-5049
note;quoted-printable:Skype: gmyrestrand=0D=0A=
MSN Messenger: [email protected]=0D=0A=

x-mozilla-html:FALSE
url:http://www.falconstor.com
version:2.1
end:vcard

#14 lore raw
Quoted 7 lines · Geir A. Myrestrand
Quoted 5 lines · Nathan Scott
Quoted 2 lines · Geir A. Myrestrand

Sorry Nathan, but I'm not sure I understand what you refer to with "test out the latest" --are you referring to the kernel or XFS?

Yes. :) Current mainline kernels, or CVS from oss.sgi.com would have this fix I'm thinking of.

Quoted 8 lines

Thanks for the "clarification"... ;-)

I'll try to update the XFS code first, hopefully it won't break too many of the patches Eric had in his source RPM...but I will try to correct them if they fail to apply cleanly...

I'll keep you posted.

I would just get kernel-2.6.19 and run the test, see if it's fixed upstream, don't worry about merging & munging xfs code from one place o another...

-Eric

#15 lore raw
Quoted 6 lines · Eric Sandeen

I would just get kernel-2.6.19 and run the test, see if it's fixed upstream, don't worry about merging & munging xfs code from one place o another...

-Eric

I have SLES 10 (SUSE Linux Enterprise Server) installed on the same box,
the kernel is 2.6.16.21-0.8-smp. Would it be worth checking that one
first?

Signature · 1 line
Geir A. Myrestrand
#16 lore raw
Quoted 11 lines · Geir A. Myrestrand
Quoted 5 lines · Eric Sandeen

I would just get kernel-2.6.19 and run the test, see if it's fixed upstream, don't worry about merging & munging xfs code from one place o another...

-Eric

I have SLES 10 (SUSE Linux Enterprise Server) installed on the same box,
the kernel is 2.6.16.21-0.8-smp. Would it be worth checking that one
first?

Sure.

-Eric

#17 lore raw
Quoted 7 lines · Eric Sandeen
Quoted 6 lines · Geir A. Myrestrand
Quoted 5 lines · Eric Sandeen

I would just get kernel-2.6.19 and run the test, see if it's fixed upstream, don't worry about merging & munging xfs code from one place o another...

-Eric

Quoted 8 lines
Quoted 4 lines

I have SLES 10 (SUSE Linux Enterprise Server) installed on the same box,
the kernel is 2.6.16.21-0.8-smp. Would it be worth checking that one
first?

Sure.

-Eric

O'boy, looks like it happens on SLES10 as well. It froze on the first XFS freeze call, it does not return.

strace -ff -o freeze-1.txt xfs_freeze -f /mnt/xfs
\_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs

A sync call hangs too. Looks like I need to test an even newer kernel...

Signature · 1 line
Geir A. Myrestrand
#18 lore raw
Quoted 8 lines · Geir A. Myrestrand

O'boy, looks like it happens on SLES10 as well. It froze on the first XFS freeze call, it does not return.

strace -ff -o freeze-1.txt xfs_freeze -f /mnt/xfs
\_ /bin/sh -f /usr/sbin/xfs_freeze -f /mnt/xfs
\_ /usr/sbin/xfs_io -r -p xfs_freeze -x -c freeze /mnt/xfs

A sync call hangs too. Looks like I need to test an even newer kernel...

You don't have multiple concurrent freezes happening do you....

-Eric

#19 lore raw
Quoted 5 lines · Eric Sandeen

You don't have multiple concurrent freezes happening do you....

-Eric

Nope, I don't need to. Now it happens on the first call to xfs_freeze. Sometimes in the past I did freeze, unfreeze, and then freeze before it happened, but this was still serialized from the same terminal window.

Oh, could strace be the reason as to why it now always happens on the first invocation of the freeze command? Maybe it affects the wait calls... Well, without strace it would probably just happen on the second invocation of freeze (after unfreeze) like it did on RHELAS4U2...

My test case is very simple, the only thing special is that I simplified my test case so much that I end up using a file as my disk (setup as a loopback device). However, the problem was initially discovered using a real disk, so I don't think that is the problem.

If you're interested I can forward the test case involving probably less than 10 commands and a simple Perl script of maybe 20 lines.

Signature · 1 line
Geir A. Myrestrand
#21 lore raw
Quoted 7 lines · James Braid
Quoted 2 lines

http://sandeen.net/rhel4_xfs/kernel-module-xfs-2.6.9-42.0.2.EL-0.2-1.src.rpm

Has anyone had any luck with getting NFS exports to work with these updated modules? I'm not having any luck, and just getting "Permission denied" errors when trying to mount an NFS exported filesystem using these modules. Works fine with the stock modules from a rebuilt RHEL4 kernel.

That's odd. You have the module on the server, exporting an xfs filesystem, and you're getting permission denied on the client?

-Eric

#22 lore raw
Quoted 2 lines · Eric Sandeen

That's odd. You have the module on the server, exporting an xfs filesystem, and you're getting permission denied on the client?

Yep. And rmmod'ing the updated XFS module and insmod'ing the older module makes it work again.

(yeah, in an ideal world we'd just run the latest kernel, but we're stuck on RHEL4 for a bunch of reasons and XFS is so much better than ext3 for our purposes, even with the old crufty version in the RHEL4 kernel)

#23 lore raw
Quoted 6 lines · James Braid
Quoted 2 lines · Eric Sandeen

That's odd. You have the module on the server, exporting an xfs filesystem, and you're getting permission denied on the client?

Yep. And rmmod'ing the updated XFS module and insmod'ing the older module makes it work again.

Interesting - what are the exact versions of "updated" and "older" that you are using? I'll look at the diff...

Thanks,
-Eric

Quoted 5 lines

(yeah, in an ideal world we'd just run the latest kernel, but we're stuck on RHEL4 for a bunch of reasons and XFS is so much better than ext3 for our purposes, even with the old crufty version in the RHEL4 kernel)