Skip to contentSkip to conversation

Invisible files and input/output errors

DISCUSSION Christoph Bartoschek 2 participants 5 messages Dec 3, 2007 – Dec 6, 2007 2 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

we use a DFS-NFS-Gateway on AIX 5.1 as our NFS-Server. This works fine most of the time but for four months we have strange problems. There are basically two issues:

1. Using "ls" in specific directories after certain idle time gives us an "Input/Output Error" without any results.

2. In some directories some entries are not shown. For example there is a directory with the subdirectory ak_fcp in it. "ls" does not show this directory. But it is possible to do "cd ak_fcp" and then normally work with it. It is also possible to access files in the directory while being in the parent directory: " less ak_fcp/logfile" works.

Does anybody know how to get rid of the problems? The problems occur on several different linux versions.

Greetings
Christoph

#1 lore raw
Quoted 9 lines · On Tuesday December 4, [email protected] wrote:

Hi,

we use a DFS-NFS-Gateway on AIX 5.1 as our NFS-Server. This works fine most of the time but for four months we have strange problems. There are basically two issues:

1. Using "ls" in specific directories after certain idle time gives us an "Input/Output Error" without any results.

Quoted 8 lines

2. In some directories some entries are not shown. For example there is a directory with the subdirectory ak_fcp in it. "ls" does not show this directory. But it is possible to do "cd ak_fcp" and then normally work with it. It is also possible to access files in the directory while being in the parent directory: " less ak_fcp/logfile" works.

Does anybody know how to get rid of the problems? The problems occur on several different linux versions.

Sounds a lot like directory cookie problems.

Can you get a network trace of the NFS packets.
Something like

tcpdump -s 0 -w /tmp/trace host NAME-OF-SERVER

then cause the failure while that is running.
Either compress and attach the result to an Email, or if it is too big, stick it on a website somewhere.

NeilBrown

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw
Quoted 11 lines

Sounds a lot like directory cookie problems.

Can you get a network trace of the NFS packets.
Something like

tcpdump -s 0 -w /tmp/trace host NAME-OF-SERVER

then cause the failure while that is running.
Either compress and attach the result to an Email, or if it is too big, stick it on a website somewhere.

I have attached a trace of a "ls" that does not show some directories.

I have to make some tests to reproduce the input/output errors.

Greetings
Christoph

#4 lore raw
Quoted 6 lines · On Tuesday December 4, [email protected] wrote:
Quoted 6 lines

Sounds a lot like directory cookie problems.

Can you get a network trace of the NFS packets.
Something like

Quoted 7 lines
Quoted 5 lines

tcpdump -s 0 -w /tmp/trace host NAME-OF-SERVER

then cause the failure while that is running.
Either compress and attach the result to an Email, or if it is too big, stick it on a website somewhere.

I have attached a trace of a "ls" that does not show some directories.

There is one interesting feature of this trace.

The 'cookie' assigned to each directory entry normally is 32 more than the previous entry, though in some places the gap is 64 or another large multiple of 32.

The whole directory is fetched in three requests - each request sending the cookie from the last name in the previous reply.

For the last request, the cookie for the first entry returned is 992 more than that last cookie in the previous reply.
This is by far that largest gap, and that fact that it aligns with a break between two requests seems a little suspicious.

It might be interesting to remove some of the files that appear early in the list, e.g.

COUNT8SF manga.tcl test-run tanga.tcl flex sopec mopec thumper

and see how that affects the returned list of files.

In any case, it looks from the trace that the Linux client so doing the right thing based on the information returned, so I would suggested reporting this to whoever provided your NFS-DFS gateway.

NeilBrown

A message that is not in the archive (sent off-list or not yet received): 18263.13509.397157.489566-wvvUuzkyo1EYVZTmpyfIwg@public.gmane.org

#6 lore raw
Quoted 4 lines

In any case, it looks from the trace that the Linux client so doing the right thing based on the information returned, so I would suggested reporting this to whoever provided your NFS-DFS gateway.

Thanks, I am going to contact the server guys.

Christoph