Skip to contentSkip to conversation

Custom hardware Qdisc...

DISCUSSION David Daney 2 participants 4 messages Dec 11, 2009 – Dec 12, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Our hardware (Octeon SOC) has fairly flexible packet output queuing that can be done completely in hardware.

I am thinking of trying to take advantage of this in the driver (currently in drivers/staging/octeon).

At a very hige level my tentative plan is to implement a custom Qdisc that has an enqueue method that places the packets directly in the output queues after they are classified.

The dequeue method would be a noop returning NULL because once the packets are on the queues, no further action is needed from the driver.

Does doing something like this seem plausible? If not, what are the main problems I might be overlooking?

Thanks in advance,
David Daney

maintainer
#1 lore raw

From: David Daney <[email protected]>
Date: Fri, 11 Dec 2009 10:26:08 -0800

Quoted 2 lines

Our hardware (Octeon SOC) has fairly flexible packet output queuing that can be done completely in hardware.

You haven't described sufficiently what your hardware is capable of.

Generally, implementing qdiscs specifically to support hardware features is not how things are handled.

But really we need to know a lot more about the details of your hardware to make any kind of real suggestions.

#2 lore raw
Quoted 9 lines · David Miller

From: David Daney <[email protected]>
Date: Fri, 11 Dec 2009 10:26:08 -0800

Quoted 2 lines

Our hardware (Octeon SOC) has fairly flexible packet output queuing that can be done completely in hardware.

You haven't described sufficiently what your hardware is capable of.

Up to 16 output queues per port. With (almost) any combination of absolute and weighted priorities between the various queues.

Quoted 3 lines

Generally, implementing qdiscs specifically to support hardware features is not how things are handled.

I think I could emulate the default pfifo Qdisk fairly well in hardware.
By making it a Qdisc, the user would still be able to switch to one of
the software implementations if desired at runtime.

Quoted 3 lines

But really we need to know a lot more about the details of your hardware to make any kind of real suggestions.

Well see above for the overview. I just wanted to get an idea if such an approach would be highly frowned upon before coding anything up.

We see significant cacheline bouncing in the Qdisc code and can increase
packet forwarding rates by 10%-15% by setting the txqueuelen to zero
to disable the queuing. I would like to see if the same thing can be achieved without throwing away the features of the Qdisc.

David Daney

maintainer
#3 lore raw

From: David Daney <[email protected]>
Date: Fri, 11 Dec 2009 16:58:50 -0800

Quoted 4 lines
Quoted 3 lines · David Miller

From: David Daney <[email protected]>
Date: Fri, 11 Dec 2009 10:26:08 -0800

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Our hardware (Octeon SOC) has fairly flexible packet output queuing that can be done completely in hardware.

You haven't described sufficiently what your hardware is capable of.

Up to 16 output queues per port. With (almost) any combination of absolute and weighted priorities between the various queues.

It sounds like these queues could be configured to have equal priorities and thus be used as purely a TX multiqueue scheme for better scalability.

I suspect that doing that alone will fix the qdisc cache line bouncing you mention.