Skip to contentSkip to conversation

CIFS VFS: Send error in Flush = -9

DISCUSSION Jeff Berman 3 participants 5 messages Jan 31, 2014 – Feb 6, 2014 7 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi there,

I'm getting lots of CIFS errors in my system log:

Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
 in Flush = -9
Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
 in Flush = -9
Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
 in Flush = -9

Usually 10-20 messages will appear at once like this. This had happened occasionally when I was on Ubuntu 12.04 LTS, but since upgrading to Ubuntu 13.10 they are appearing daily (I actually upgraded to get a newer version of cifs-utils in the hopes that the problem would go away).

Modinfo cifs shows cifs.ko at version 2.01, and cifs-utils is at 6.0.

The share is mounted via fstab:
//server2/share /media/tm cifs credentials=/home/jeff/.smbcredentials, _netdev,uid=jeff,gid=jeff,file_mode=0777,dir_mode=0777,iocharset=utf8 ,sec=ntlm 0 0

I increased logging to level "3" and here is what's going on when the first Flush error occurs (sorry about the formatting, Gmane is limiting each line to 80 chars):

Jan 31 06:17:36 Server1 kernel: [70838.026873] /build/buildd/linux-3.11.0/
fs/cifs/connect.c: RFC1002 header 0x23
Jan 31 06:17:36 Server1 kernel: [70838.026879] /build/buildd/linux-3.11.0/
fs/cifs/misc.c: checkSMB Length: 0x27, smb_buf_length: 0x23
Jan 31 06:17:36 Server1 kernel: [70838.026889] /build/buildd/linux-3.11.0/
fs/cifs/transport.c: cifs_sync_mid_result: cmd=5 mid=34092 state=4
Jan 31 06:17:36 Server1 kernel: [70838.026893] /build/buildd/linux-3.11.0/
fs/cifs/file.c: CIFS VFS: leaving cifs_strict_fsync (xid = 2694478) rc = 0
Jan 31 06:17:36 Server1 kernel: [70838.027602] /build/buildd/linux-3.11.0/
fs/cifs/file.c: CIFS VFS: in cifs_strict_fsync as Xid: 2694479 with uid: 1000
Jan 31 06:17:36 Server1 kernel: [70838.027606] /build/buildd/linux-3.11.0/
fs/cifs/file.c: Sync file - name: 1c datasync: 0x0
Jan 31 06:17:36 Server1 kernel: [70838.027610] /build/buildd/linux-3.11.0/
fs/cifs/cifssmb.c: In CIFSSMBFlush
Jan 31 06:17:36 Server1 kernel: [70838.027614] /build/buildd/linux-3.11.0/
fs/cifs/transport.c: For smb_command 5
Jan 31 06:17:36 Server1 kernel: [70838.027618] /build/buildd/linux-3.11.0/
fs/cifs/transport.c: Sending smb: smb_len=37
Jan 31 06:17:36 Server1 kernel: [70838.027875] /build/buildd/linux-3.11.0/
fs/cifs/connect.c: RFC1002 header 0x23
Jan 31 06:17:36 Server1 kernel: [70838.027882] /build/buildd/linux-3.11.0/
fs/cifs/misc.c: checkSMB Length: 0x27, smb_buf_length: 0x23
Jan 31 06:17:36 Server1 kernel: [70838.027896] /build/buildd/linux-3.11.0/
fs/cifs/transport.c: cifs_sync_mid_result: cmd=5 mid=34093 state=4
Jan 31 06:17:36 Server1 kernel: [70838.027900] Status code returned 0xc0000008 
NT_STATUS_INVALID_HANDLE
Jan 31 06:17:36 Server1 kernel: [70838.027906] /build/buildd/linux-3.11.0/
fs/cifs/netmisc.c: Mapping smb error code 0xc0000008 to POSIX err -9
Jan 31 06:17:36 Server1 kernel: [70838.027910] CIFS VFS: Send error in Flush = -9
Jan 31 06:17:36 Server1 kernel: [70838.027915] /build/buildd/linux-3.11.0/
fs/cifs/file.c: CIFS VFS: leaving cifs_strict_fsync (xid = 2694479) rc = -9
Jan 31 06:17:36 Server1 kernel: [70838.027981] /build/buildd/linux-3.11.0/
fs/cifs/xattr.c: CIFS VFS: in cifs_getxattr as Xid: 2694480 with uid: 1000
Jan 31 06:17:36 Server1 kernel: [70838.027985] /build/buildd/linux-3.11.0/
fs/cifs/dir.c: name: \afpd.log
Jan 31 06:17:36 Server1 kernel: [70838.027989] /build/buildd/linux-3.11.0/
fs/cifs/xattr.c: Security xattr namespace not supported yet
Jan 31 06:17:36 Server1 kernel: [70838.027992] /build/buildd/linux-3.11.0/

fs/cifs/xattr.c: CIFS VFS: leaving cifs_getxattr (xid = 2694480) rc = -95

Any suggestions of how to work around this or how I could provide more info?

Thanks!

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw

-9 is EBADF.

If you can send a wireshark trace of traffic between client and server that would be helpful.

Quoted 76 lines · Jeff Berman

Hi there,

I'm getting lots of CIFS errors in my system log:

Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
 in Flush = -9
Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
 in Flush = -9
Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
 in Flush = -9

Usually 10-20 messages will appear at once like this. This had happened occasionally when I was on Ubuntu 12.04 LTS, but since upgrading to Ubuntu 13.10 they are appearing daily (I actually upgraded to get a newer version of cifs-utils in the hopes that the problem would go away).

Modinfo cifs shows cifs.ko at version 2.01, and cifs-utils is at 6.0.

The share is mounted via fstab:
//server2/share /media/tm cifs credentials=/home/jeff/.smbcredentials, _netdev,uid=jeff,gid=jeff,file_mode=0777,dir_mode=0777,iocharset=utf8 ,sec=ntlm 0 0

I increased logging to level "3" and here is what's going on when the first Flush error occurs (sorry about the formatting, Gmane is limiting each line to 80 chars):

Jan 31 06:17:36 Server1 kernel: [70838.026873] /build/buildd/linux-3.11.0/
fs/cifs/connect.c: RFC1002 header 0x23
Jan 31 06:17:36 Server1 kernel: [70838.026879] /build/buildd/linux-3.11.0/
fs/cifs/misc.c: checkSMB Length: 0x27, smb_buf_length: 0x23
Jan 31 06:17:36 Server1 kernel: [70838.026889] /build/buildd/linux-3.11.0/
fs/cifs/transport.c: cifs_sync_mid_result: cmd=5 mid=34092 state=4
Jan 31 06:17:36 Server1 kernel: [70838.026893] /build/buildd/linux-3.11.0/
fs/cifs/file.c: CIFS VFS: leaving cifs_strict_fsync (xid = 2694478) rc = 0
Jan 31 06:17:36 Server1 kernel: [70838.027602] /build/buildd/linux-3.11.0/
fs/cifs/file.c: CIFS VFS: in cifs_strict_fsync as Xid: 2694479 with uid: 1000
Jan 31 06:17:36 Server1 kernel: [70838.027606] /build/buildd/linux-3.11.0/
fs/cifs/file.c: Sync file - name: 1c datasync: 0x0
Jan 31 06:17:36 Server1 kernel: [70838.027610] /build/buildd/linux-3.11.0/
fs/cifs/cifssmb.c: In CIFSSMBFlush
Jan 31 06:17:36 Server1 kernel: [70838.027614] /build/buildd/linux-3.11.0/
fs/cifs/transport.c: For smb_command 5
Jan 31 06:17:36 Server1 kernel: [70838.027618] /build/buildd/linux-3.11.0/
fs/cifs/transport.c: Sending smb: smb_len=37
Jan 31 06:17:36 Server1 kernel: [70838.027875] /build/buildd/linux-3.11.0/
fs/cifs/connect.c: RFC1002 header 0x23
Jan 31 06:17:36 Server1 kernel: [70838.027882] /build/buildd/linux-3.11.0/
fs/cifs/misc.c: checkSMB Length: 0x27, smb_buf_length: 0x23
Jan 31 06:17:36 Server1 kernel: [70838.027896] /build/buildd/linux-3.11.0/
fs/cifs/transport.c: cifs_sync_mid_result: cmd=5 mid=34093 state=4
Jan 31 06:17:36 Server1 kernel: [70838.027900] Status code returned 0xc0000008
NT_STATUS_INVALID_HANDLE
Jan 31 06:17:36 Server1 kernel: [70838.027906] /build/buildd/linux-3.11.0/
fs/cifs/netmisc.c: Mapping smb error code 0xc0000008 to POSIX err -9
Jan 31 06:17:36 Server1 kernel: [70838.027910] CIFS VFS: Send error in Flush = -9
Jan 31 06:17:36 Server1 kernel: [70838.027915] /build/buildd/linux-3.11.0/
fs/cifs/file.c: CIFS VFS: leaving cifs_strict_fsync (xid = 2694479) rc = -9
Jan 31 06:17:36 Server1 kernel: [70838.027981] /build/buildd/linux-3.11.0/
fs/cifs/xattr.c: CIFS VFS: in cifs_getxattr as Xid: 2694480 with uid: 1000
Jan 31 06:17:36 Server1 kernel: [70838.027985] /build/buildd/linux-3.11.0/
fs/cifs/dir.c: name: \afpd.log
Jan 31 06:17:36 Server1 kernel: [70838.027989] /build/buildd/linux-3.11.0/
fs/cifs/xattr.c: Security xattr namespace not supported yet
Jan 31 06:17:36 Server1 kernel: [70838.027992] /build/buildd/linux-3.11.0/

fs/cifs/xattr.c: CIFS VFS: leaving cifs_getxattr (xid = 2694480) rc = -95

Any suggestions of how to work around this or how I could provide more info?

Thanks!

--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): CADT32e+5whrPysC+8JmNupP1WPS1dT6hbb8O+DrH=SYSpsLk4w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#4 lore raw
Quoted 7 lines

From: Shirish Pargaonkar <[email protected]>

To: Jeff Berman <spamact-/[email protected]>
Cc: linux-cifs <[email protected]>
Sent: Friday, January 31, 2014 12:07 PM
Subject: Re: CIFS VFS: Send error in Flush = -9

Quoted 18 lines

-9 is EBADF.

If you can send a wireshark trace of traffic between client and server that would be helpful.

Quoted 10 lines · Jeff Berman

Hi there,

I'm getting lots of CIFS errors in my system log:

 Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
  in Flush = -9
 Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
  in Flush = -9
 Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error

  in Flush = -9

Ok, I think I did the wireshark trace correctly.  It's attached to bug 10417 (https://bugzilla.samba.org/show_bug.cgi?id=10417).

Thank you!

A message that is not in the archive (sent off-list or not yet received): 1391592105.31475.YahooMailNeo-GNkJ2jOaCZKvrfWm4H71L5EhsgyP+Z75VpNB7YpNyf8@public.gmane.org

#6 lore raw
Quoted 27 lines · On Wed, 5 Feb 2014 01:21:45 -0800 (PST) Jeff Berman…
Quoted 7 lines

From: Shirish Pargaonkar <[email protected]>

To: Jeff Berman <spamact-/[email protected]>
Cc: linux-cifs <[email protected]>
Sent: Friday, January 31, 2014 12:07 PM
Subject: Re: CIFS VFS: Send error in Flush = -9

Quoted 18 lines

-9 is EBADF.

If you can send a wireshark trace of traffic between client and server that would be helpful.

Quoted 10 lines · Jeff Berman

Hi there,

I'm getting lots of CIFS errors in my system log:

 Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
  in Flush = -9
 Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error
  in Flush = -9
 Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS VFS: Send error

  in Flush = -9

Quoted 3 lines

Ok, I think I did the wireshark trace correctly.  It's attached to bug 10417 (https://bugzilla.samba.org/show_bug.cgi?id=10417).

Thank you!

Ok, I took a look and I'm pretty sure the problem is the long-standing bug in how the client tried to use virtual circuits. Applying this patch from mainline kernels should help things:

commit 9ae6cf606a33b0a762798df0fb742848bcc685b5

Author: Jeff Layton <[email protected]>
Date:   Mon Sep 16 11:23:45 2013 -0400

cifs: stop trying to use virtual circuits

Please let us know if it doesn't.

Signature · 1 line
Jeff Layton <[email protected]>

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): 1391652130.90245.YahooMailNeo-GNkJ2jOaCZKvrfWm4H71L5EhsgyP+Z75VpNB7YpNyf8@public.gmane.org

#11 lore raw
Quoted 8 lines · On Wed, 5 Feb 2014 18:02:10 -0800 (PST) Jeff Berman…

Hi guys,

I installed a 3.12.9 kernel, but the "Send error in Flush = -9" errors still appear in quantity.  Any other ideas?

Thanks,

Jeff

(cc'ing linux-cifs -- sorry, I don't do private support unless you're
paying me ;)

What would be best is to reopen the bug and attach a new capture if you're still seeing the problem. Are there multiple machines involved that are connecting through any sort of NAT? If so, then they'll all need the same patch...

Quoted 119 lines

----- Original Message -----

Quoted 117 lines

From: Jeff Layton <[email protected]>
To: Jeff Berman <spamact-/[email protected]>
Cc: Shirish Pargaonkar <[email protected]>
Sent: Wednesday, February 5, 2014 10:58 AM
Subject: Re: CIFS VFS: Send error in Flush = -9

Quoted 3 lines · On Wed, 5 Feb 2014 10:45:16 -0800 (PST) Jeff Berman…

Hi Jeff,

Thanks for your reply!  What you said makes sense to me.  I'd like to

apply the patch but after researching kernel patches for a couple hours I realize that I'm in over my head.  I have no problem attempting it, but I can't figure out how to tell if my current kernel contains it already or how to install it (and any needed prerequisites).  My Xubuntu installation is at "Ubuntu 3.11.0-15.25-generic 3.11.10".

Quoted 2 lines

You're obviously super busy, but can you suggest a link with relevant

instructions to follow?  There are hundreds of different kernel guides out there dating back over a decade.

Quoted 5 lines

Thanks again!

Jeff

I'm pretty sure that patch went into mainline in v3.12, so it's likely your v3.11 kernel doesn't have it. The trivial way to get it would be to just move to a v3.12 kernel.

If that's not possible you'll probably just need to get the sources for your kernel apply the patch to it and build it (I assume it should just apply directly with little or no merge conflicts)...or talk the ubuntu folks into doing it for you.

Good luck!

Quoted 5 lines

----- Original Message -----

Quoted 3 lines

From: Jeff Layton <[email protected]>
To: Jeff Berman <spamact-/[email protected]>
Cc: linux-cifs <[email protected]>; Shirish Pargaonkar

<[email protected]>

Quoted 9 lines
Quoted 9 lines

Sent: Wednesday, February 5, 2014 3:00 AM
Subject: Re: CIFS VFS: Send error in Flush = -9

On Wed, 5 Feb 2014 01:21:45 -0800 (PST)

Quoted 2 lines · Jeff Berman

  > From: Shirish Pargaonkar <[email protected]>

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

  > To: Jeff Berman <spamact-/[email protected]> > Cc: linux-cifs <[email protected]> > Sent: Friday, January 31, 2014 12:07 PM
  > Subject: Re: CIFS VFS: Send error in Flush = -9
  >
  > -9 is EBADF.
  >
  > If you can send a wireshark trace of traffic between client

and server

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

  > that would be helpful.
  >
  >
  >
  > On Fri, Jan 31, 2014 at 1:20 PM, Jeff Berman

<spamact-/[email protected]>

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines · wrote:

  >>  Hi there,
  >>
  >>  I'm getting lots of CIFS errors in my system log:
  >>
  >>  Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS

VFS: Send

Quoted 3 lines
Quoted 3 lines

error

Quoted 2 lines

  >>   in Flush = -9
  >>  Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS

VFS: Send

Quoted 3 lines
Quoted 3 lines

error

Quoted 2 lines

  >>   in Flush = -9
  >>  Jan 30 08:46:05 Server1 kernel: [747319.264235] CIFS

VFS: Send

Quoted 5 lines
Quoted 5 lines

error

Quoted 4 lines

  >>   in Flush = -9

  Ok, I think I did the wireshark trace correctly.  It's

attached to bug

Quoted 5 lines
Quoted 5 lines

10417 (https://bugzilla.samba.org/show_bug.cgi?id=10417).

Quoted 2 lines

  Thank you!

Ok, I took a look and I'm pretty sure the problem is the

long-standing

Quoted 16 lines
Quoted 16 lines

bug in how the client tried to use virtual circuits. Applying this patch from mainline kernels should help things:

commit 9ae6cf606a33b0a762798df0fb742848bcc685b5
Author: Jeff Layton <[email protected]> Date:   Mon Sep 16 11:23:45 2013 -0400

    cifs: stop trying to use virtual circuits

Please let us know if it doesn't.

Signature · 1 line
Jeff Layton <[email protected]
Quoted 1 line
Signature · 1 line
Jeff Layton <[email protected]>
Signature · 1 line
Jeff Layton <[email protected]>