NFSv4 client locks up on larger writes with Kerberos enabled
Subsystems (2)
Bruce,
Yes. It’s a Dell EMC Unity 300 storage appliance.
From what I have gathered, under the hood it is running SLES12SP1
3.12.74-60.64.66.1.NEOKERNEL_SLES12SP1 #2 SMP Fri May 17 06;41:36 EDT 2019 x86_64
Unfortunately I only have limited access to the appliance through a “service” account. I do not have root privileges and I can only run certain commands under the service account.
I’ve got a ticket open with them and it’s been escalated to their “engineering” department but they haven’t provided much insight as of yet.
If you give me some idea on what might help I’ll try to get more info if I can.
-Kevin
Quoted 40 lines · Bruce Fields
Quoted 11 lines · Chuck Lever
Quoted 8 lines
Quoted 1 line · Bruce Fields
Quoted 3 lines · Chuck Lever
Sounds like the NFS server is dropping the connection. With GSS enabled, that's usually a sign that the GSS window has overflowed.
Would that show up in the rpc statistics on the client somehow?
More likely on the server. The client just sees a disconnect without any explanation attached.
So watching a count of disconnects might at least tell us something?
Quoted 3 lines
gss_verify_headeris where the checking is done on the server. Disappointingly, I see some dprintk's in there, but no static trace events.Kevin, was this a Linux server?
--b.
Quoted 15 lines
Quoted 7 lines
In that case--I seem to remember there's a way to configure the size of the client's slot table, maybe lowering that (decreasing the number of rpc's allowed to be outstanding at a time) would work around the problem.
Should the client be doing something different to avoid or recover from overflows of the gss window?
The client attempts to meter the request stream so that it stays within the bounds of the GSS sequence number window. The stream of requests is typically unordered coming out of the transmit queue.
There is some new code (since maybe v5.0?) that handles the
metering:gss_xmit_need_reencode().