Skip to contentSkip to conversation

ENOBUFS and dev_queue_xmit()

DISCUSSION Alex Pankratov 2 participants 3 messages Jun 15, 2004

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I've been poking around rather weird problem today where send() on UDP socket was failing with ENETDOWN. When traced down to the kernel, it happened to originate in dev_queue_xmit() -

	if (!netif_queue_stopped(dev)) {
		...
	}
	...
	$here

meaning that the device's queue was stopped. The comment there implies that only a broken virtual device may end up $here, but the fact is that I saw it for a physical interface running slightly modified via-rhine driver. The original driver code contains a snippet that stops the queue if its Tx ring buffer becomes full -

	if (np->cur_tx == np->dirty_tx + TX_QUEUE_LEN)
		netif_stop_queue(dev);

and this code got actually executed with a hack applied.

At this point I thought that dev_queue_xmit() must be mistakenly returning ENETDOWN instead of ENOBUFS, but looking at 'man 2 send' I saw that -

ENOBUFS
The output queue for a network interface was full...snip... (Normally, this does not occur in Linux. Packets are just silently dropped when a device queue overflows.)

Hmmm... so I looked at e100 and tulip and these both stop the queue too if they run out of buffers. In other words, the 'silently dropped' clause from man page does not seem to be consistently followed.

Is this a known (pseudo?) issue ? ENOBUFS makes much more sense in this context. I can certainly check interface status (IFF_UP) on every ENETDOWN to see what's the real cause, but that's kind of ugly.

Alex

#1 lore raw
Quoted 4 lines · Alex Pankratov

I've been poking around rather weird problem today where send() on UDP socket was failing with ENETDOWN. When traced down to the kernel, it happened to originate in dev_queue_xmit() -

Quoted 8 lines
	if (!netif_queue_stopped(dev)) {
		...
	}
	...
	$here

meaning that the device's queue was stopped. The comment there implies that only a broken virtual device may end up $here,

How did you end up there with a real phy device?? Are you trying to circumvent the qdisc subsystem? If yes, you are responsible for how all this gets handled.

Quoted 20 lines

but
the fact is that I saw it for a physical interface running slightly modified via-rhine driver. The original driver code contains a snippet that stops the queue if its Tx ring buffer becomes full -

	if (np->cur_tx == np->dirty_tx + TX_QUEUE_LEN)
		netif_stop_queue(dev);

and this code got actually executed with a hack applied.

At this point I thought that dev_queue_xmit() must be mistakenly returning ENETDOWN instead of ENOBUFS, but looking at 'man 2 send' I saw that -

ENOBUFS
The output queue for a network interface was full...snip...
(Normally, this does not occur in Linux. Packets are just silently dropped when a device queue overflows.)

Quoted 8 lines

Hmmm... so I looked at e100 and tulip and these both stop the queue too if they run out of buffers. In other words, the 'silently dropped' clause from man page does not seem to be consistently followed.

Is this a known (pseudo?) issue ? ENOBUFS makes much more sense in this context. I can certainly check interface status (IFF_UP) on every ENETDOWN to see what's the real cause, but that's kind of ugly.

Did you mean when no space is left in the ring? Thats different from ENOBUFF. If not, not sure i see how a driver xmit path gets involved in kmallocing.
Look at the return code the driver returns. In case of a full ring, it should return a busy signal and the top layer will retry later. You dont have to worry about any of that if you are running the std linux semantics, of course. I have a feeling you have attempted to bypass it otherwise the question becomes: how you even ended in this situation.

cheers,
jamal

#2 lore raw
Quoted 3 lines · jamal
Quoted 1 line · Alex Pankratov

I've been poking around rather weird problem today where send()

[snip]

Quoted 7 lines
Quoted 2 lines

meaning that the device's queue was stopped. The comment there implies that only a broken virtual device may end up $here,

How did you end up there with a real phy device?? Are you trying to circumvent the qdisc subsystem? If yes, you are responsible for how all this gets handled.

Now that I looked at dev.c code again I ask myself the very same questions :) I am not circumventing qdisc, at least intentionally, and I don't do anything fancy with dev->qdisc. It must be a bug due to some of my changes, ignore the original question. Thanks for your help.

Quoted 6 lines
Quoted 5 lines

Is this a known (pseudo?) issue ? ENOBUFS makes much more sense in this context. I can certainly check interface status (IFF_UP) on every ENETDOWN to see what's the real cause, but that's kind of ugly.

Quoted 9 lines

Did you mean when no space is left in the ring? Thats different
from ENOBUFF. If not, not sure i see how a driver xmit path gets
involved in kmallocing.
Look at the return code the driver returns. In case of a full ring, it
should return a busy signal and the top layer will retry later.
You dont have to worry about any of that if you are running the std linux semantics, of course. I have a feeling you have attempted to bypass it otherwise the question becomes: how you even ended in this
situation.

I'm pretty sure you're right about breaking the semantics. I'll check it out, and re-complain if it's not my problem. Thanks again.