Skip to contentSkip to conversation

Introducing P4TC

PATCH Superseded Jamal Hadi Salim 3 participants 42 messages Jan 22, 2024 – Feb 21, 2024 1 message not in the archive

Subsystems (6)

v10 of 16 Superseded Latest is v16 →

lore
Share & tools
lore

Jamal Hadi Salim

author maintainer
#0 lore raw

v10 cover Introducing P4TC

What is P4?
-----------

The Programming Protocol-independent Packet Processors (P4) is an open source, domain-specific programming language for specifying data plane behavior.

The current P4 landscape includes an extensive range of deployments, products, projects and services, etc[9][12]. Two major NIC vendors, Intel[10] and AMD[11] currently offer P4-native NICs. P4 is currently curated by the Linux Foundation[9].

On why P4 - see small treatise here:[4].

What is P4TC?
-------------

P4TC is a net-namespace aware P4 implementation over TC; meaning, a P4 program and its associated objects and state are attachend to a kernel _netns_ structure. IOW, if we had two programs across netns' or within a netns they have no visibility to each others objects (unlike for example TC actions whose kinds are "global" in nature or eBPF maps visavis bpftool).

P4TC builds on top of many years of Linux TC experiences of a netlink control path interface coupled with a software datapath with an equivalent offloadable hardware datapath. In this patch series we are focussing only on the s/w datapath. The s/w and h/w path equivalence that TC provides is relevant for a primary use case of P4 where some (currently) large consumers of NICs provide vendors their datapath specs in P4. In such a case one could generate specified datapaths in s/w and test/validate the requirements before hardware acquisition.

Unlike other approaches like TC Flower which require kernel and user space changes when new datapath objects like packet headers are introduced P4TC, with these patches, provides _kernel and user space code change independence_. Meaning:
A P4 program describes headers, parsers, etc alongside the datapath processing; the compiler uses the P4 program as input and generates several artifacts which are then loaded into the kernel to manifest the intended datapath. In addition to the generated datapath, control path constructs are generated. The process is described further below in "P4TC Workflow".

There have been many discussions and meetings since within the community since about 2015 in regards to P4 over TC[2] and we are finally proving (if the drama ends soon) to the naysayers that we do get stuff done!

A lot more of the P4TC motivation is captured at:
https://github.com/p4tc-dev/docs/blob/main/why-p4tc.md

__P4TC Architecture__

The current architecture was described at netdevconf 0x17[14] and if you prefer academic conference papers, a short paper is available here[15].

There are 4 parts:

1) A Template CRUD provisioning API for manifesting a P4 program and its associated objects in the kernel. The template provisioning API uses netlink. See patch description further down..

2) A Runtime CRUD+ API code which is used for controlling the different runtime behavior of the P4 objects. The runtime API uses netlink. See notes further down. See patch description further down..

3) P4 objects and their control interfaces: tables, actions, externs, etc. Any object that requires control plane interaction resides in the TC domain and is subject to the runtime API. The intended goal is to make use of the tc semantics of skip_sw/hw to target P4 program objects either in s/w or h/w.

4) S/W Datapath code hooks. The s/w datapath is eBPF based and is generated by a compiler based on the P4 spec. When accessing any P4 object that requires control plane interfaces, the eBPF code accesses the P4TC side from #3 above using kfuncs.

The generated eBPF code is derived from [13] with enhancements and fixes to meet our requirements (see "Restating Our Requirements" further below).

__P4TC Workflow__

The workflow is as follows:

1) A developer writes a P4 program, "myprog"

2) Compiles it using the P4C compiler[8]. The compiler generates 3 outputs:
a) A shell script which form template definitions for the different P4 objects "myprog" utilizes (tables, externs, actions etc).
b) the parser and the rest of the datapath are generated as eBPF and need to be compiled into binaries. At the moment the parser and the main control block are generated as separate eBPF program but this could change in the future (without affecting any kernel code).
c) A json introspection file used for the control plane (by iproute2/tc).

3) At this point the artifacts from #2 could be handed to an operator. Either the operator is handed an ebpf binary or source which they compile at this point.
The operator executes the shell script(s) to manifest the functional "myprog" into the kernel.

4) The operator instantiates "myprog" pipeline via the tc P4 filter to ingress/egress (depending on P4 arch) of one or more netdevs/ports (illustrated below as "block 22"). Note the tc filter here is used to manage the P4 program pipeline which may include pieces in h/w, tc(eBPF) and XDP(eBPF).

Several ways to instantiate:

Example1: parser is an action:
"tc filter add block 22 ingress protocol all prio 10 p4 pname myprog \
action bpf obj $PARSER.o section p4parser/tc-ingress \
action bpf obj $PROGNAME.o section p4prog/tc"

Example2: parser explicitly bound and rest of dpath as an action:
"tc filter add block 22 ingress protocol all prio 10 p4 pname myprog \
prog tc obj $PARSER.o section p4parser/tc-ingress \
action bpf obj $PROGNAME.o section p4prog/tc"

Example3: parser is at XDP, rest of dpath as an action:
"tc filter add block 22 ingress protocol all prio 10 p4 pname myprog \
prog type xdp obj $PARSER.o section p4parser/xdp-ingress \
pinned_link /path/to/xdp-prog-link \
action bpf obj $PROGNAME.o section p4prog/tc"

Example4: parser+prog at XDP:
"tc filter add block 22 ingress protocol all prio 10 p4 pname myprog \
prog type xdp obj $PROGNAME.o section p4prog/xdp \
pinned_link /path/to/xdp-prog-link"

See individual patches for more examples tc vs xdp etc. Also see section on "challenges" (further below on this cover letter).

Once "myprog" P4 program is instantiated one can start updating table entries and/or creating actions at runtime.
Example, creating an entry matched with a dstAddr 10.0.1.2/32 in myprog's table named "mytable" which redirect to eno1:

  tc p4ctrl create myprog/table/mytable dstAddr 10.0.1.2/32 \
    action send_to_port param port eno1

__P4TC Runtime Control Path__

The control interface builds on past experience and tries to get things right from the beginning (example filtering is separated from depending on existing object TLVs and made generic); also the code is written in such a way it is mostly lockless.

The P4TC control interface, using netlink, provides what we call a CRUDPS abstraction which stands for: Create, Read(get), Update, Delete, Subscribe, Publish. From a high level PoV the following describes a conformant high level API (both on netlink data model and code level):

Create(</path/to/object, DATA>+) Read(</path/to/object>, [optional filter]) Update(</path/to/object>, DATA>+) Delete(</path/to/object>, [optional filter]) Subscribe(</path/to/object>, [optional filter])

Note, we _dont_ treat "dump" or "flush" as speacial. If "path/to/object" points to a table then a "Delete" implies "flush" and a "Read" implies dump but if it points to an entry (by specifying a key) then "Delete" implies deleting and entry and "Read" implies reading that single entry. It should be noted that both "Delete" and "Read" take an optional filter parameter. The filter can define further refinements to what the control plane wants read or deleted. "Subscribe" uses built in netlink event management. It, as well, takes a filter which can further refine what events get generated to the control plane (taken out of this patchset, to be re-added with consideration of [16]).

Lets show some samples:

..create an entry
tc p4ctrl create myprog/table/mytable \
dstAddr 10.0.1.2/32 action send_to_port param port eno1

..Batch create entries
tc p4ctrl create myprog/table/mytable \
entry dstAddr 10.1.1.2/32 action send_to_port param port eno1 \
entry dstAddr 10.1.10.2/32 action send_to_port param port eno10 \
entry dstAddr 10.0.2.2/32 action send_to_port param port eno2

..Get an entry (note "read" is interchangeably used as "get" which is a common
semantic in tc):
tc p4ctrl read myprog/table/mytable \
dstAddr 10.0.2.2/32

..dump mytable
tc p4ctrl read myprog/table/mytable

..dump mytable for all entries whose key fits within 10.1.0.0/16
tc p4ctrl read myprog/table/mytable \
filter key/myprog/mytable/dstAddr = 10.1.0.0/16

..dump all mytable entries which have an action send_to_port with param "eno1"
tc p4ctrl get myprog/table/mytable \
filter param/act/myprog/send_to_port/port = "eno1"

The filter expression is powerful, f.e you could say:

  tc p4ctrl get myprog/table/mytable \
  filter param/act/myprog/send_to_port/port = "eno1" && \
         key/myprog/mytable/dstAddr = 10.1.0.0/16

It also works on built in metadata, example in the following case dumping entries from mytable that have seen activity in the last 10 secs:
tc p4ctrl get myprog/table/mytable \
filter msecs_since < 10000

Delete follows the same syntax as get/read, so for sake of brevity we won't show more example than how to flush mytable:

tc p4ctrl delete myprog/table/mytable

Mystery question: How do we achieve iproute2-kernel independence and how does "tc p4ctrl" as a cli know how to program the kernel given an arbitrary command line as shown above? Answer(s): It queries the compiler generated json file in #2c above. The json file has enough details to figure out that we have a program called "myprog" which has a table "mytable" that has a key name "dstAddr" which happens to be type ipv4 address prefix. The json file also provides details to show that the table "mytable" supports an action called "send_to_port" which accepts a parameter "port" of type netdev (see the types patch for all supported P4 data types). All P4 components have names, IDs, and types - so this makes it very easy to map into netlink. Once user space tc/p4ctrl validates the human command input, it creates standard binary netlink structures (TLVs etc) which are sent to the kernel. See the runtime table entry patch for more details.

__P4TC Datapath__

The P4TC s/w datapath execution is generated in eBPF. Any objects that require control interfacing reside in the "P4TC domain" and are controlled via netlink as described above. Per packet execution and state and even objects that do not require control interfacing (like the P4 parser) are generated as eBPF.

A packet arriving on ingress of any of the ports on block 22 will first be exercised via the (generated eBPF) parser component to extract the headers (the ip destination address in labelled "dstAddr" above).
The datapath then proceeds to use "dstAddr", table ID and pipeline ID as a key to do a lookup in myprog's "mytable" which returns the action params which are then used to execute the action in the eBPF datapath (eventually sending out packets to eno1).
On a table miss, mytable's default miss action (not described) is executed.

__Description of Patches__

P4TC is designed to have no impact on the core code for other users of the kernel. IOW, you either can compile it out (or even compile it in and if you dont use it) then there should be no impact on your workload performance or functionality.

We do make small kernel changes (only on tc code, and not intrusive at all in the first 5 patches). Patch #1 adds infrastructure for P4 actions that can be created on as need basis for the P4 program requirement. This patch makes a small incision into act_api which shouldn't affect the performance (or functionality) of the existing actions. Patches 2-4 are minimalist enablers for P4TC and have no effect the classical tc action.
Patch 5 adds infrastructure support for preallocation of dynamic actions.

The core P4TC code implements several P4 objects.

1) Patch #6 introduces P4 data types which are consumed by the rest of the code
2) Patch #7 introduces the templating API. i.e CRUD commands for templates
3) Patch #8 introduces the concept of templating Pipelines. i.e CRUD commands for P4 pipelines.
4) Patch #9 introduces the action templates and associated CRUD commands.
5) Patch #10 introduce the action runtime infrastructure.
6) Patch #11 introduces the concept of P4 table templates and associated CRUD commands for tables.
7) Patch #12 introduces runtime table entry infra and associated CU commands.
8) Patch #13 introduces runtime table entry infra and associated RD commands.
9) Patch #14 introduces interaction of eBPF to P4TC tables via kfunc.
10) Patch #15 introduces the TC classifier P4 used at runtime.

Note, to have the minimal viable implementation we need to have extern patch(es) on top of these. There are a few more patches (2-5) not in this patchset that deal with P4 Extern objects. So consider this patchset to be "part 1"; "part2" will come later. Now if we could only push the trivial patches 1-5... then we can fit all in the max allowed 15.

__Testing__

Speaking of testing - we have ~300 tdc test cases. This number is growing as we are adjusting to accommodate for eBPF.
These tests are run on our CICD system on pull requests and after commits are approved. The CICD does a lot of other tests (more since v2, thanks to Simon's input)including:
checkpatch, sparse, smatch, coccinelle, 32 bit and 64 bit builds tested on both X86, ARM 64 and emulated BE via qemu s390. We trigger performance testing in the CICD to catch performance regressions (currently only on the control path, but in the future for the datapath).
Syzkaller runs 24/7 on dedicated hardware, originally we focussed only on memory sanitizer but recently added support for concurrency sanitizer. Before main releases we ensure each patch will compile on its own to help in git bisect and run the xmas tree tool. We eventually put the code via coverity.

In addition we are working on a tool that will take a P4 program, run it through the compiler, and generate permutations of traffic patterns via symbolic execution that will test both positive and negative datapath code paths. The test generator tool is still work in progress and will be generated by the P4 compiler.
Note: We have other code that test parallelization etc which we are trying to find a fit for in the kernel tree's testing infra.

__Restating Our Requirements__

Given this code is not intrusive at all, it is amazing the kind of noise these patches have generated;-> We would like to emphasize that _we see eBPF as infrastructure tooling available to us and not the end goal_. Please help us with technical input on for example how we can do better kfuncs, etc. If you want to critique, then our requirements should be your guide and please be considerate that this is about P4, not eBPF. IOW:
We would appreciate technical commentary instead of bikeshedding on how _you_ would have implemented this probably with more eBPF. Moreover, it is sad to see there was zero input from anyone in the eBPF world for 7 RFC postings. If i am ranting here is because we have spent over a year now on this topic - we have taken the initial input and have given you eBPF. So lets make progress please.

The initial release was presented in October 2022[20] and RFC in January/2023 had a "scriptable" datapath (the idea built on the u32 classifier[17] and pedit action[18] approach). Post RFC V1, we made changes to fit the feedback to integrate eBPF to replace the "scriptable" software datapath. On our part, the goal for the change was to meet folks in the middle as a compromise. No regrets on the journey since after all the effort because we ended getting XDP which was not in the original picture. Some of our efforts are captured at [1][3] and in the patch history.

In this section we review the original scriptable version against the current implementation which uses eBPF and in the process re-enumerate our requirements.

To be very clear: Our intention for P4TC is to target _the TC crowd_. Essentially developers and ops people already familiar and deploying TC based infra.
More importantly the original intent for P4TC was to enable _ops folks_ more than devs (given code is being generated and doesn't need humans to write it).

With TC, we gain the whole "familiar" package of match-action pipeline abstraction++, meaning from the control plane(see discussion above) all the way to the tooling infra, i.e iproute2/tc cli, netlink infra interface (request/response, event subscribe/multicast-publish, congestion control etc), s/w and h/w symbiosis, the autonomous kernel control, etc.
The main advantage over vendor specific implementations(which is the current alternative) is: with P4TC we have a singular vendor-neutral interface via the kernel using well understood mechanisms that have gained learnings from deployment experience.

So lets list some of these requirements and compare whether moving to eBPF affected us or gave us an advantage.

0) Understood Control Plane semantics

This requirement is unaffected.
The control plane remains as netlink and therefore we get the classical multi-user CRUD+Publish/subscribe APIs built in.

1) Must support SW/HW equivalence

This requirement is unaffected. The control plane is netlink. Any semantics to select between sw and hw via skip_sw/hw semantics is maintained.

2) Supporting expressibility of the universe set of P4 progs

It is a must to support 100% of all possible P4 programs. In the past the eBPF verifier, for example in [13], had to be worked around and even then there are cases where we couldnt avoid path explosion when branching is involved and failed to run. So we were skeptical about using eBPF to begin with. Kfuncs changed our minds. Note, there are still challenges running all potential P4 programs at the XDP level - but the pipeline could be split between XDP and TC in such cases. The compiler can be told to generate pieces that run on XDP and other on TC (see examples).
Summary: This requirement is unaffected.

3) Operational usability

By maintaining the TC control plane (even in presence of eBPF datapath) runtime aspects remain unchanged. So for our target audience of folks who have deployed tc, including offloads, the comfort zone is unchanged.

There is some loss in operational usability because we now have more knobs:
the extra compilation, loading and syncing of ebpf binaries, etc. IOW, I can no longer just ship someone a shell script(ascii) in an email to someone and say "go run this and "myprog" will just work".

4) Operational and development Debuggability

If something goes wrong, the tc craftsperson is now required to have additional knowledge of eBPF code and process.
Our intent is to compensate this challenge with debug tools that ease the craftperson's debugging.

5) Opportunity for rapid prototyping of new ideas

This is not exactly a requirement but something that became a useful feature during the P4TC development phase. When the compiler was lagging behind in features was to often handcode the template scripts.
Then you would dump back the template from the kernel and do a diff to ensure the kernel didn't get something wrong. Essentially, this was a nice debug feature. During development, we wrote scripts that covered a range of P4 architectures(PSA, V1, etc) which required no kernel code changes.

Over time the debug feature morphed into: a) start by handcoding scripts then
b) read it back and then c) generate the P4 code.
It means one could start with the template scripts outside of the constraints of a P4 architecture spec(PNA/PSA) or even within a P4 architecture then test some ideas and eventually feed back the concepts to the compiler authors or modify or create a new P4 architecture and share with the P4 standards folks.

To summarize in presence of eBPF: The debugging idea is probably still alive. One could dump, with proper tooling(bpftool for example), the loaded eBPF code and be able to check for differences. But this is not the interesting part. The concept of going back from whats in the kernel to P4 is a lot more difficult to implement mostly due to scoping of DSL vs general purpose. It may be lost. We have been thinking of ways to use BTF and embedding annotations in the eBPF code and binary but more thought is required and we welcome suggestions.

6) Supporting per namespace program

In P4TC every program and its associated objects have unique IDs which are generated by the compiler. Multiple or the same P4 program(s) can run independently in different namespaces alongside their appropriate state and object instance parameterization (despite name or ID collission). This requirement is still met (by virtue of keeping P4 program control objects within the TC domain and attaching to a netns).

__Challenges__

1) Concept of tc block in XDP is _very tedious_ to implement. It would be nice if we can use concept there as well, since we expect P4 to work with many ports. It will likely work now with patch [19].

2) Right now we are using "packed" construct to enforce alignment in kfunc data exchange; but we're wondering if there is potential to use BTF to understand parameters and their offsets and encode this information at the compiler level.

3) At the moment we are creating a static buffer of 128B to retrieve the action parameters. If you have a lot of table entries and individual(non-shared) action instances with actions that require very little (or no) param space a lot of memory is wasted. There may also be cases where 128B may not be enough; (likely this is something we can teach the P4C compiler). If we can have dynamic pointers instead for kfunc fixed length parameterization then this issue is resolvable.

4) See "Restating Our Requirements" #5.
We would really appreciate ideas/suggestions, etc.

__References__

[1]https://github.com/p4tc-dev/docs/blob/main/p4-conference-2023/2023P4WorkshopP4TC.pdf [2]https://github.com/p4tc-dev/docs/blob/main/why-p4tc.md#historical-perspective-for-p4tc [3]https://2023p4workshop.sched.com/event/1KsAe/p4tc-linux-kernel-p4-implementation-approaches-and-evaluation [4]https://github.com/p4tc-dev/docs/blob/main/why-p4tc.md#so-why-p4-and-how-does-p4-help-here [5]https://lore.kernel.org/netdev/[email protected]/T/#mf59be7abc5df3473cff3879c8cc3e2369c0640a6 [6]https://lore.kernel.org/netdev/[email protected]/T/#m783cfd79e9d755cf0e7afc1a7d5404635a5b1919 [7]https://lore.kernel.org/netdev/[email protected]/T/#ma8c84df0f7043d17b98f3d67aab0f4904c600469 [8]https://github.com/p4lang/p4c/tree/main/backends/tc
[9]https://p4.org/ [10]https://www.intel.com/content/www/us/en/products/details/network-io/ipu/e2000-asic.html [11]https://www.amd.com/en/accelerators/pensando
[12]https://github.com/sonic-net/DASH/tree/main
[13]https://github.com/p4lang/p4c/tree/main/backends/ebpf [14]https://netdevconf.info/0x17/sessions/talk/integrating-ebpf-into-the-p4tc-datapath.html [15]https://dl.acm.org/doi/10.1145/3630047.3630193 [16]https://lore.kernel.org/netdev/[email protected]/ [17.a]https://netdevconf.info/0x13/session.html?talk-tc-u-classifier
[17.b]man tc-u32
[18]man tc-pedit
[19] https://lore.kernel.org/netdev/[email protected]/T/#m86e71743d1d83b728bb29d5b877797cb4942e835
[20.a] https://netdevconf.info/0x16/sessions/talk/your-network-datapath-will-be-p4-scripted.html
[20.b] https://netdevconf.info/0x16/sessions/workshop/p4tc-workshop.html

--------
HISTORY
--------

Changes in Version 10
----------------------
1) A couple of patches from the earlier version were clean enough to submit, so we did. This gave us room to split the two largest patches each into two. Even though the split is not git-bisactable and really some of it didn't make much sense (eg spliting a create, and update in one patch and delete and get into another) we made sure each of the split patches compiled independently. The idea is to reduce the number of lines of code to review and when we get sufficient reviews we will put the splits together again. See patch #12 and #13 as well as patches #7 and #8).

2) Add more context in patch 0. Please READ!

3) Added dump/delete filters back to the code - we had taken them out in the earlier patches to reduce the amount of code for review - but in retrospect we feel they are important enough to push earlier rather than later.

Changes In version 9
---------------------

1) Remove the largest patch (externs) to ease review.

2) Break up action patches into two to ease review bringing down the patches that need more scrutiny to 8 (the first 7 are almost trivial).

3) Fixup prefix naming convention to p4tc_xxx for uapi and p4a_xxx for actions to provide consistency(Jiri).

4) Silence sparse warning "was not declared. Should it be static?" for kfuncs by making them static. TBH, not sure if this is the right solution but it makes sparse happy and hopefully someone will comment.

Changes In Version 8
---------------------

1) Fix all the patchwork warnings and improve our ci to catch them in the future

2) Reduce the number of patches to basic max(15) to ease review.

Changes In Version 7
-------------------------

0) First time removing the RFC tag!

1) Removed XDP cookie. It turns out as was pointed out by Toke(Thanks!) - that using bpf links was sufficient to protect us from someone replacing or deleting a eBPF program after it has been bound to a netdev.

2) Add some reviewed-bys from Vlad.

3) Small bug fixes from v6 based on testing for ebpf.

4) Added the counter extern as a sample extern. Illustrating this example because it is slightly complex since it is possible to invoke it directly from the P4TC domain (in case of direct counters) or from eBPF (indirect counters). It is not exactly the most efficient implementation (a reasonable counter impl should be per-cpu).

Changes In RFC Version 6
-------------------------

1) Completed integration from scriptable view to eBPF. Completed integration of externs integration.

2) Small bug fixes from v5 based on testing.

Changes In RFC Version 5
-------------------------

1) More integration from scriptable view to eBPF. Small bug fixes from last integration.

2) More streamlining support of externs via kfunc (create-on-miss, etc)

3) eBPF linking for XDP.

There is more eBPF integration/streamlining coming (we are getting close to conversion from scriptable domain).

Changes In RFC Version 4
-------------------------

1) More integration from scriptable to eBPF. Small bug fixes.

2) More streamlining support of externs via kfunc (one additional kfunc).

3) Removed per-cpu scratchpad per Toke's suggestion and instead use XDP metadata.

There is more eBPF integration coming. One thing we looked at but is not in this patchset but should be in the next is use of eBPF link in our loading (see "challenge #1" further below).

Changes In RFC Version 3
-------------------------

These patches are still in a little bit of flux as we adjust to integrating eBPF. So there are small constructs that are used in V1 and 2 but no longer used in this version. We will make a V4 which will remove those. The changes from V2 are as follows:

1) Feedback we got in V2 is to try stick to one of the two modes. In this version we are taking one more step and going the path of mode2 vs v2 where we had 2 modes.

2) The P4 Register extern is no longer standalone. Instead, as part of integrating into eBPF we introduce another kfunc which encapsulates Register as part of the extern interface.

3) We have improved our CICD to include tools pointed to us by Simon. See "Testing" further below. Thanks to Simon for that and other issues he caught. Simon, we discussed on issue [7] but decided to keep that log since we think it is useful.

4) A lot of small cleanups. Thanks Marcelo. There are two things we need to re-discuss though; see: [5], [6].

5) We removed the need for a range of IDs for dynamic actions. Thanks Jakub.

6) Clarify ambiguity caused by smatch in an if(A) else if(B) condition. We are guaranteed that either A or B must exist; however, lets make smatch happy. Thanks to Simon and Dan Carpenter.

Changes In RFC Version 2
-------------------------

Version 2 is the initial integration of the eBPF datapath.
We took into consideration suggestions provided to use eBPF and put effort into analyzing eBPF as datapath which involved extensive testing.
We implemented 6 approaches with eBPF and ran performance analysis and presented our results at the P4 2023 workshop in Santa Clara[see: 1, 3] on each of the 6 vs the scriptable P4TC and concluded that 2 of the approaches are sensible (4 if you account for XDP or TC separately).

Conclusions from the exercise: We lose the simple operational model we had prior to integrating eBPF. We do gain performance in most cases when the datapath is less compute-bound.
For more discussion on our requirements vs journeying the eBPF path please scroll down to "Restating Our Requirements" and "Challenges".

This patch set presented two modes.
mode1: the parser is entirely based on eBPF - whereas the rest of the SW datapath stays as _scriptable_ as in Version 1.
mode2: All of the kernel s/w datapath (including parser) is in eBPF.

The key ingredient for eBPF, that we did not have access to in the past, is kfunc (it made a big difference for us to reconsider eBPF).

In V2 the two modes are mutually exclusive (IOW, you get to choose one or the other via Kconfig).

Jamal Hadi Salim (15):
net: sched: act_api: Introduce P4 actions list
net/sched: act_api: increase action kind string length net/sched: act_api: Update tc_action_ops to account for P4 actions net/sched: act_api: add struct p4tc_action_ops as a parameter to
lookup callback
net: sched: act_api: Add support for preallocated P4 action instances
p4tc: add P4 data types
p4tc: add template API
p4tc: add template pipeline create, get, update, delete
p4tc: add template action create, update, delete, get, flush and dump
p4tc: add runtime action support
p4tc: add template table create, update, delete, get, flush and dump
p4tc: add runtime table entry create and update
p4tc: add runtime table entry get, delete, flush and dump
p4tc: add set of P4TC table kfuncs
p4tc: add P4 classifier

 include/linux/bitops.h            |    1 +
 include/net/act_api.h             |   23 +-
 include/net/p4tc.h                |  642 ++++++
 include/net/p4tc_types.h          |   91 +
 include/net/tc_act/p4tc.h         |   52 +
 include/uapi/linux/p4tc.h         |  433 ++++
 include/uapi/linux/pkt_cls.h      |   19 +
 include/uapi/linux/rtnetlink.h    |   18 +
 include/uapi/linux/tc_act/tc_p4.h |   11 +
 net/sched/Kconfig                 |   23 +
 net/sched/Makefile                |    3 +
 net/sched/act_api.c               |  190 +-
 net/sched/cls_api.c               |    2 +-
 net/sched/cls_p4.c                |  450 +++++
 net/sched/p4tc/Makefile           |    8 +
 net/sched/p4tc/p4tc_action.c      | 2305 ++++++++++++++++++++++
 net/sched/p4tc/p4tc_bpf.c         |  338 ++++
 net/sched/p4tc/p4tc_filter.c      |  872 +++++++++
 net/sched/p4tc/p4tc_pipeline.c    |  678 +++++++
 net/sched/p4tc/p4tc_runtime_api.c |  145 ++
 net/sched/p4tc/p4tc_table.c       | 1779 +++++++++++++++++
 net/sched/p4tc/p4tc_tbl_entry.c   | 3044 +++++++++++++++++++++++++++++
 net/sched/p4tc/p4tc_tmpl_api.c    |  609 ++++++
 net/sched/p4tc/p4tc_types.c       | 1287 ++++++++++++
 net/sched/p4tc/trace.c            |   10 +
 net/sched/p4tc/trace.h            |   44 +
 security/selinux/nlmsgtab.c       |   10 +-
 27 files changed, 13052 insertions(+), 35 deletions(-)
 create mode 100644 include/net/p4tc.h
 create mode 100644 include/net/p4tc_types.h
 create mode 100644 include/net/tc_act/p4tc.h
 create mode 100644 include/uapi/linux/p4tc.h
 create mode 100644 include/uapi/linux/tc_act/tc_p4.h
 create mode 100644 net/sched/cls_p4.c
 create mode 100644 net/sched/p4tc/Makefile
 create mode 100644 net/sched/p4tc/p4tc_action.c
 create mode 100644 net/sched/p4tc/p4tc_bpf.c
 create mode 100644 net/sched/p4tc/p4tc_filter.c
 create mode 100644 net/sched/p4tc/p4tc_pipeline.c
 create mode 100644 net/sched/p4tc/p4tc_runtime_api.c
 create mode 100644 net/sched/p4tc/p4tc_table.c
 create mode 100644 net/sched/p4tc/p4tc_tbl_entry.c
 create mode 100644 net/sched/p4tc/p4tc_tmpl_api.c
 create mode 100644 net/sched/p4tc/p4tc_types.c
 create mode 100644 net/sched/p4tc/trace.c
 create mode 100644 net/sched/p4tc/trace.h
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v10 1/15 net: sched: act_api: Introduce P4 actions list

In P4 we require to generate new actions "on the fly" based on the specified P4 action definition. P4 action kinds, like the pipeline they are attached to, must be per net namespace, as opposed to native action kinds which are global. For that reason, we chose to create a separate structure to store P4 actions.

---
 include/net/act_api.h |   8 ++-
 net/sched/act_api.c   | 123 +++++++++++++++++++++++++++++++++++++-----
 net/sched/cls_api.c   |   2 +-
 3 files changed, 116 insertions(+), 17 deletions(-)
include/net/act_api.h +6 -2
index e1e5e72b9..ab28c2254 100644
@@ -105,6 +105,7 @@ typedef void (*tc_action_priv_destructor)(void *priv);
 
 struct tc_action_ops {
 	struct list_head head;
+	struct list_head p4_head;
 	char    kind[IFNAMSIZ];
 	enum tca_id  id; /* identifier should match kind */
 	unsigned int	net_id;
@@ -199,8 +200,10 @@ int tcf_idr_check_alloc(struct tc_action_net *tn, u32 *index,
 int tcf_idr_release(struct tc_action *a, bool bind);
 
 int tcf_register_action(struct tc_action_ops *a, struct pernet_operations *ops);
+int tcf_register_p4_action(struct net *net, struct tc_action_ops *act);
 int tcf_unregister_action(struct tc_action_ops *a,
 			  struct pernet_operations *ops);
+void tcf_unregister_p4_action(struct net *net, struct tc_action_ops *act);
 int tcf_action_destroy(struct tc_action *actions[], int bind);
 int tcf_action_exec(struct sk_buff *skb, struct tc_action **actions,
 		    int nr_actions, struct tcf_result *res);
@@ -208,8 +211,9 @@ int tcf_action_init(struct net *net, struct tcf_proto *tp, struct nlattr *nla,
 		    struct nlattr *est,
 		    struct tc_action *actions[], int init_res[], size_t *attr_size,
 		    u32 flags, u32 fl_flags, struct netlink_ext_ack *extack);
-struct tc_action_ops *tc_action_load_ops(struct nlattr *nla, u32 flags,
-					 struct netlink_ext_ack *extack);
+struct tc_action_ops *
+tc_action_load_ops(struct net *net, struct nlattr *nla,
+		   u32 flags, struct netlink_ext_ack *extack);
 struct tc_action *tcf_action_init_1(struct net *net, struct tcf_proto *tp,
 				    struct nlattr *nla, struct nlattr *est,
 				    struct tc_action_ops *a_o, int *init_res,
net/sched/act_api.c +109 -14

215 lines not on this page · raw message

net/sched/cls_api.c +1 -1
index 92a12e3d0..2fec3f80b 100644
@@ -3323,7 +3323,7 @@ int tcf_exts_validate_ex(struct net *net, struct tcf_proto *tp, struct nlattr **
 			struct tc_action_ops *a_o;
 
 			flags |= TCA_ACT_FLAGS_POLICE | TCA_ACT_FLAGS_BIND;
-			a_o = tc_action_load_ops(tb[exts->police], flags,
+			a_o = tc_action_load_ops(net, tb[exts->police], flags,
 						 extack);
 			if (IS_ERR(a_o))
 				return PTR_ERR(a_o);
Signature · 1 line
2.34.1
Reviewed-by on v10 1/15
#2 lore raw

Re: [PATCH v10 net-next 01/15] net: sched: act_api: Introduce P4 actions list

Quoted 6 lines · Jamal Hadi Salim

In P4 we require to generate new actions "on the fly" based on the specified P4 action definition. P4 action kinds, like the pipeline they are attached to, must be per net namespace, as opposed to native action kinds which are global. For that reason, we chose to create a separate structure to store P4 actions.

Quoted 6 lines
Quoted 288 lines
---
 include/net/act_api.h |   8 ++-
 net/sched/act_api.c   | 123 +++++++++++++++++++++++++++++++++++++-----
 net/sched/cls_api.c   |   2 +-
 3 files changed, 116 insertions(+), 17 deletions(-)
include/net/act_api.h +6 -2
index e1e5e72b9..ab28c2254 100644
@@ -105,6 +105,7 @@ typedef void (*tc_action_priv_destructor)(void *priv);
 
 struct tc_action_ops {
 	struct list_head head;
+	struct list_head p4_head;
 	char    kind[IFNAMSIZ];
 	enum tca_id  id; /* identifier should match kind */
 	unsigned int	net_id;
@@ -199,8 +200,10 @@ int tcf_idr_check_alloc(struct tc_action_net *tn, u32 *index,
 int tcf_idr_release(struct tc_action *a, bool bind);
 
 int tcf_register_action(struct tc_action_ops *a, struct pernet_operations *ops);
+int tcf_register_p4_action(struct net *net, struct tc_action_ops *act);
 int tcf_unregister_action(struct tc_action_ops *a,
 			  struct pernet_operations *ops);
+void tcf_unregister_p4_action(struct net *net, struct tc_action_ops *act);
 int tcf_action_destroy(struct tc_action *actions[], int bind);
 int tcf_action_exec(struct sk_buff *skb, struct tc_action **actions,
 		    int nr_actions, struct tcf_result *res);
@@ -208,8 +211,9 @@ int tcf_action_init(struct net *net, struct tcf_proto *tp, struct nlattr *nla,
 		    struct nlattr *est,
 		    struct tc_action *actions[], int init_res[], size_t *attr_size,
 		    u32 flags, u32 fl_flags, struct netlink_ext_ack *extack);
-struct tc_action_ops *tc_action_load_ops(struct nlattr *nla, u32 flags,
-					 struct netlink_ext_ack *extack);
+struct tc_action_ops *
+tc_action_load_ops(struct net *net, struct nlattr *nla,
+		   u32 flags, struct netlink_ext_ack *extack);
 struct tc_action *tcf_action_init_1(struct net *net, struct tcf_proto *tp,
 				    struct nlattr *nla, struct nlattr *est,
 				    struct tc_action_ops *a_o, int *init_res,
net/sched/act_api.c +109 -14
index 3e30d7260..e4a1b8f5a 100644
@@ -57,6 +57,40 @@ static void tcf_free_cookie_rcu(struct rcu_head *p)
 	kfree(cookie);
 }
 
+static unsigned int p4_act_net_id;
+
+struct tcf_p4_act_net {
+	struct list_head act_base;
+	rwlock_t act_mod_lock;
+};
+
+static __net_init int tcf_p4_act_base_init_net(struct net *net)
+{
+	struct tcf_p4_act_net *p4_base_net = net_generic(net, p4_act_net_id);
+
+	INIT_LIST_HEAD(&p4_base_net->act_base);
+	rwlock_init(&p4_base_net->act_mod_lock);
+
+	return 0;
+}
+
+static void __net_exit tcf_p4_act_base_exit_net(struct net *net)
+{
+	struct tcf_p4_act_net *p4_base_net = net_generic(net, p4_act_net_id);
+	struct tc_action_ops *ops, *tmp;
+
+	list_for_each_entry_safe(ops, tmp, &p4_base_net->act_base, p4_head) {
+		list_del(&ops->p4_head);
+	}
+}
+
+static struct pernet_operations tcf_p4_act_base_net_ops = {
+	.init = tcf_p4_act_base_init_net,
+	.exit = tcf_p4_act_base_exit_net,
+	.id = &p4_act_net_id,
+	.size = sizeof(struct tc_action_ops),
+};
+
 static void tcf_set_action_cookie(struct tc_cookie __rcu **old_cookie,
 				  struct tc_cookie *new_cookie)
 {
@@ -962,6 +996,48 @@ static void tcf_pernet_del_id_list(unsigned int id)
 	mutex_unlock(&act_id_mutex);
 }
 
+static struct tc_action_ops *tc_lookup_p4_action(struct net *net, char *kind)
+{
+	struct tcf_p4_act_net *p4_base_net = net_generic(net, p4_act_net_id);
+	struct tc_action_ops *a, *res = NULL;
+
+	read_lock(&p4_base_net->act_mod_lock);
+	list_for_each_entry(a, &p4_base_net->act_base, p4_head) {
+		if (strcmp(kind, a->kind) == 0) {
+			if (try_module_get(a->owner))
+				res = a;
+			break;
+		}
+	}
+	read_unlock(&p4_base_net->act_mod_lock);
+
+	return res;
+}
+
+void tcf_unregister_p4_action(struct net *net, struct tc_action_ops *act)
+{
+	struct tcf_p4_act_net *p4_base_net = net_generic(net, p4_act_net_id);
+
+	write_lock(&p4_base_net->act_mod_lock);
+	list_del(&act->p4_head);
+	write_unlock(&p4_base_net->act_mod_lock);
+}
+EXPORT_SYMBOL(tcf_unregister_p4_action);
+
+int tcf_register_p4_action(struct net *net, struct tc_action_ops *act)
+{
+	struct tcf_p4_act_net *p4_base_net = net_generic(net, p4_act_net_id);
+
+	if (tc_lookup_p4_action(net, act->kind))
+		return -EEXIST;
+
+	write_lock(&p4_base_net->act_mod_lock);
+	list_add(&act->p4_head, &p4_base_net->act_base);
+	write_unlock(&p4_base_net->act_mod_lock);
+
+	return 0;
+}
+
 int tcf_register_action(struct tc_action_ops *act,
 			struct pernet_operations *ops)
 {
@@ -1032,7 +1108,7 @@ int tcf_unregister_action(struct tc_action_ops *act,
 EXPORT_SYMBOL(tcf_unregister_action);
 
 /* lookup by name */
-static struct tc_action_ops *tc_lookup_action_n(char *kind)
+static struct tc_action_ops *tc_lookup_action_n(struct net *net, char *kind)
 {
 	struct tc_action_ops *a, *res = NULL;
 
@@ -1040,31 +1116,48 @@ static struct tc_action_ops *tc_lookup_action_n(char *kind)
 		read_lock(&act_mod_lock);
 		list_for_each_entry(a, &act_base, head) {
 			if (strcmp(kind, a->kind) == 0) {
-				if (try_module_get(a->owner))
-					res = a;
-				break;
+				if (try_module_get(a->owner)) {
+					read_unlock(&act_mod_lock);
+					return a;
+				}
 			}
 		}
 		read_unlock(&act_mod_lock);
+
+		return tc_lookup_p4_action(net, kind);
 	}
+
 	return res;
 }
 
 /* lookup by nlattr */
-static struct tc_action_ops *tc_lookup_action(struct nlattr *kind)
+static struct tc_action_ops *tc_lookup_action(struct net *net,
+					      struct nlattr *kind)
 {
+	struct tcf_p4_act_net *p4_base_net = net_generic(net, p4_act_net_id);
 	struct tc_action_ops *a, *res = NULL;
 
 	if (kind) {
 		read_lock(&act_mod_lock);
 		list_for_each_entry(a, &act_base, head) {
+			if (nla_strcmp(kind, a->kind) == 0) {
+				if (try_module_get(a->owner)) {
+					read_unlock(&act_mod_lock);
+					return a;
+				}
+			}
+		}
+		read_unlock(&act_mod_lock);
+
+		read_lock(&p4_base_net->act_mod_lock);
+		list_for_each_entry(a, &p4_base_net->act_base, p4_head) {
 			if (nla_strcmp(kind, a->kind) == 0) {
 				if (try_module_get(a->owner))
 					res = a;
 				break;
 			}
 		}
-		read_unlock(&act_mod_lock);
+		read_unlock(&p4_base_net->act_mod_lock);
 	}
 	return res;
 }
@@ -1324,8 +1417,9 @@ void tcf_idr_insert_many(struct tc_action *actions[], int init_res[])
 	}
 }
 
-struct tc_action_ops *tc_action_load_ops(struct nlattr *nla, u32 flags,
-					 struct netlink_ext_ack *extack)
+struct tc_action_ops *
+tc_action_load_ops(struct net *net, struct nlattr *nla,
+		   u32 flags, struct netlink_ext_ack *extack)
 {
 	bool police = flags & TCA_ACT_FLAGS_POLICE;
 	struct nlattr *tb[TCA_ACT_MAX + 1];
@@ -1356,7 +1450,7 @@ struct tc_action_ops *tc_action_load_ops(struct nlattr *nla, u32 flags,
 		}
 	}
 
-	a_o = tc_lookup_action_n(act_name);
+	a_o = tc_lookup_action_n(net, act_name);
 	if (a_o == NULL) {
 #ifdef CONFIG_MODULES
 		bool rtnl_held = !(flags & TCA_ACT_FLAGS_NO_RTNL);
@@ -1367,7 +1461,7 @@ struct tc_action_ops *tc_action_load_ops(struct nlattr *nla, u32 flags,
 		if (rtnl_held)
 			rtnl_lock();
 
-		a_o = tc_lookup_action_n(act_name);
+		a_o = tc_lookup_action_n(net, act_name);
 
 		/* We dropped the RTNL semaphore in order to
 		 * perform the module load.  So, even if we
@@ -1477,7 +1571,7 @@ int tcf_action_init(struct net *net, struct tcf_proto *tp, struct nlattr *nla,
 	for (i = 1; i <= TCA_ACT_MAX_PRIO && tb[i]; i++) {
 		struct tc_action_ops *a_o;
 
-		a_o = tc_action_load_ops(tb[i], flags, extack);
+		a_o = tc_action_load_ops(net, tb[i], flags, extack);
 		if (IS_ERR(a_o)) {
 			err = PTR_ERR(a_o);
 			goto err_mod;
@@ -1683,7 +1777,7 @@ static struct tc_action *tcf_action_get_1(struct net *net, struct nlattr *nla,
 	index = nla_get_u32(tb[TCA_ACT_INDEX]);
 
 	err = -EINVAL;
-	ops = tc_lookup_action(tb[TCA_ACT_KIND]);
+	ops = tc_lookup_action(net, tb[TCA_ACT_KIND]);
 	if (!ops) { /* could happen in batch of actions */
 		NL_SET_ERR_MSG(extack, "Specified TC action kind not found");
 		goto err_out;
@@ -1731,7 +1825,7 @@ static int tca_action_flush(struct net *net, struct nlattr *nla,
 
 	err = -EINVAL;
 	kind = tb[TCA_ACT_KIND];
-	ops = tc_lookup_action(kind);
+	ops = tc_lookup_action(net, kind);
 	if (!ops) { /*some idjot trying to flush unknown action */
 		NL_SET_ERR_MSG(extack, "Cannot flush unknown TC action");
 		goto err_out;
@@ -2184,7 +2278,7 @@ static int tc_dump_action(struct sk_buff *skb, struct netlink_callback *cb)
 		return 0;
 	}
 
-	a_o = tc_lookup_action(kind);
+	a_o = tc_lookup_action(net, kind);
 	if (a_o == NULL)
 		return 0;
 
@@ -2251,6 +2345,7 @@ static int __init tc_action_init(void)
 	rtnl_register(PF_UNSPEC, RTM_GETACTION, tc_ctl_action, tc_dump_action,
 		      0);
 
+	register_pernet_subsys(&tcf_p4_act_base_net_ops);
 	return 0;
 }
 
net/sched/cls_api.c +1 -2
index 92a12e3d0..2fec3f80b 100644
@@ -3323,7 +3323,7 @@ int tcf_exts_validate_ex(struct net *net, struct tcf_proto *tp, struct nlattr **
 			struct tc_action_ops *a_o;
 
 			flags |= TCA_ACT_FLAGS_POLICE | TCA_ACT_FLAGS_BIND;
-			a_o = tc_action_load_ops(tb[exts->police], flags,
+			a_o = tc_action_load_ops(net, tb[exts->police], flags,
 						 extack);
 			if (IS_ERR(a_o))
 				return PTR_ERR(a_o);
--

2.34.1

author maintainer
#3 lore raw

PATCH v10 2/15 net/sched: act_api: increase action kind string length

In reply to Jamal Hadi Salim

Increase action kind string length from IFNAMSIZ to 64

The new P4 actions, created via templates, will have longer names of format: "pipeline_name/act_name". IFNAMSIZ is currently 16 and is most of the times undersized for the above format.
So, to conform to this new format, we increase the maximum name length and change its definition from IFNAMSIZ to ACTNAMSIZ to account for this extra string (pipeline name) and the '/' character.

---
 include/net/act_api.h        | 2 +-
 include/uapi/linux/pkt_cls.h | 1 +
 net/sched/act_api.c          | 6 +++---
 3 files changed, 5 insertions(+), 4 deletions(-)
include/net/act_api.h +1 -1
index ab28c2254..eb39a2101 100644
@@ -106,7 +106,7 @@ typedef void (*tc_action_priv_destructor)(void *priv);
 struct tc_action_ops {
 	struct list_head head;
 	struct list_head p4_head;
-	char    kind[IFNAMSIZ];
+	char    kind[ACTNAMSIZ];
 	enum tca_id  id; /* identifier should match kind */
 	unsigned int	net_id;
 	size_t	size;
include/uapi/linux/pkt_cls.h +1 -0
index ea277039f..dd313a727 100644
@@ -6,6 +6,7 @@
 #include <linux/pkt_sched.h>
 
 #define TC_COOKIE_MAX_SIZE 16
+#define ACTNAMSIZ 64
 
 /* Action attributes */
 enum {
net/sched/act_api.c +3 -3
index e4a1b8f5a..2ff61be8e 100644
@@ -476,7 +476,7 @@ static size_t tcf_action_shared_attrs_size(const struct tc_action *act)
 	rcu_read_unlock();
 
 	return  nla_total_size(0) /* action number nested */
-		+ nla_total_size(IFNAMSIZ) /* TCA_ACT_KIND */
+		+ nla_total_size(ACTNAMSIZ) /* TCA_ACT_KIND */
 		+ cookie_len /* TCA_ACT_COOKIE */
 		+ nla_total_size(sizeof(struct nla_bitfield32)) /* TCA_ACT_HW_STATS */
 		+ nla_total_size(0) /* TCA_ACT_STATS nested */
@@ -1424,7 +1424,7 @@ tc_action_load_ops(struct net *net, struct nlattr *nla,
 	bool police = flags & TCA_ACT_FLAGS_POLICE;
 	struct nlattr *tb[TCA_ACT_MAX + 1];
 	struct tc_action_ops *a_o;
-	char act_name[IFNAMSIZ];
+	char act_name[ACTNAMSIZ];
 	struct nlattr *kind;
 	int err;
 
@@ -1439,7 +1439,7 @@ tc_action_load_ops(struct net *net, struct nlattr *nla,
 			NL_SET_ERR_MSG(extack, "TC action kind must be specified");
 			return ERR_PTR(err);
 		}
-		if (nla_strscpy(act_name, kind, IFNAMSIZ) < 0) {
+		if (nla_strscpy(act_name, kind, ACTNAMSIZ) < 0) {
 			NL_SET_ERR_MSG(extack, "TC action name too long");
 			return ERR_PTR(err);
 		}
Signature · 1 line
2.34.1
#4 lore raw

Re: [PATCH v10 net-next 02/15] net/sched: act_api: increase action kind string length

net/sched/act_api.c tc_action_load_ops(struct net *net, struct nlattr *nla, · line 1445

Quoted 9 lines · Jamal Hadi Salim
net/sched/act_api.c +1 -1
@@ -1439,7 +1439,7 @@ tc_action_load_ops(struct net *net, struct nlattr *nla,
 			NL_SET_ERR_MSG(extack, "TC action kind must be specified");
 			return ERR_PTR(err);
 		}
-		if (nla_strscpy(act_name, kind, IFNAMSIZ) < 0) {
+		if (nla_strscpy(act_name, kind, ACTNAMSIZ) < 0) {
 			NL_SET_ERR_MSG(extack, "TC action name too long");
 			return ERR_PTR(err);
 		}

Subsquent lines here are:

        } else {
                if (strscpy(act_name, "police", IFNAMSIZ) < 0) {
		                                ^^^^^^^^
                        NL_SET_ERR_MSG(extack, "TC action name too long");

I know it won't make a difference in the end but it would be nice to keep it consistent.

#5 lore raw

Re: Re: [PATCH v10 net-next 02/15] net/sched: act_api: increase action kind string length

Quoted 12 lines · Marcelo Ricardo Leitner

net/sched/act_api.c tc_action_load_ops(struct net *net, struct nlattr *nla, · line 1445

Quoted 9 lines · Jamal Hadi Salim
net/sched/act_api.c +1 -1
@@ -1439,7 +1439,7 @@ tc_action_load_ops(struct net *net, struct nlattr *nla,
 			NL_SET_ERR_MSG(extack, "TC action kind must be specified");
 			return ERR_PTR(err);
 		}
-		if (nla_strscpy(act_name, kind, IFNAMSIZ) < 0) {
+		if (nla_strscpy(act_name, kind, ACTNAMSIZ) < 0) {
 			NL_SET_ERR_MSG(extack, "TC action name too long");
 			return ERR_PTR(err);
 		}

Subsquent lines here are:

Quoted 8 lines
        } else {
                if (strscpy(act_name, "police", IFNAMSIZ) < 0) {
		                                ^^^^^^^^
                        NL_SET_ERR_MSG(extack, "TC action name too long");

I know it won't make a difference in the end but it would be nice to keep it consistent.

Despite this, please add my tag in the next iteration:

author maintainer
#6 lore raw

Re: [PATCH v10 net-next 02/15] net/sched: act_api: increase action kind string length

In reply to Marcelo Ricardo Leitner
Quoted 13 lines · Marcelo Ricardo Leitner

net/sched/act_api.c tc_action_load_ops(struct net *net, struct nlattr *nla, · line 1445

Quoted 9 lines · Jamal Hadi Salim
net/sched/act_api.c +1 -1
@@ -1439,7 +1439,7 @@ tc_action_load_ops(struct net *net, struct nlattr *nla,
                      NL_SET_ERR_MSG(extack, "TC action kind must be specified");
                      return ERR_PTR(err);
              }
-             if (nla_strscpy(act_name, kind, IFNAMSIZ) < 0) {
+             if (nla_strscpy(act_name, kind, ACTNAMSIZ) < 0) {
                      NL_SET_ERR_MSG(extack, "TC action name too long");
                      return ERR_PTR(err);
              }

Subsquent lines here are:

Quoted 7 lines
        } else {
                if (strscpy(act_name, "police", IFNAMSIZ) < 0) {
                                                ^^^^^^^^
                        NL_SET_ERR_MSG(extack, "TC action name too long");

I know it won't make a difference in the end but it would be nice to keep it consistent.

Agreed. It was an oversight. Will fix it in the next version. Thanks for the rest of your reviews Marcelo!

cheers,
jamal

author maintainer
#7 lore raw

PATCH v10 3/15 net/sched: act_api: Update tc_action_ops to account for P4 actions

In reply to Jamal Hadi Salim

The initialisation of P4TC action instances require access to a struct p4tc_act (which appears in later patches) to help us to retrieve information like the P4 action parameters etc. In order to retrieve struct p4tc_act we need the pipeline name or id and the action name or id. Also recall that P4TC action IDs are P4 and are net namespace specific and not global like standard tc actions.
The init callback from tc_action_ops parameters had no way of supplying us that information. To solve this issue, we decided to create a new tc_action_ops callback (init_ops), that provies us with the tc_action_ops struct which then provides us with the pipeline and action name. In addition we add a new refcount to struct tc_action_ops called dyn_ref, which accounts for how many action instances we have of a specific action.

---
 include/net/act_api.h |  6 ++++++
 net/sched/act_api.c   | 14 +++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)
include/net/act_api.h +6 -0
index eb39a2101..bfded7ec6 100644
@@ -109,6 +109,7 @@ struct tc_action_ops {
 	char    kind[ACTNAMSIZ];
 	enum tca_id  id; /* identifier should match kind */
 	unsigned int	net_id;
+	refcount_t p4_ref;
 	size_t	size;
 	struct module		*owner;
 	int     (*act)(struct sk_buff *, const struct tc_action *,
@@ -120,6 +121,11 @@ struct tc_action_ops {
 			struct nlattr *est, struct tc_action **act,
 			struct tcf_proto *tp,
 			u32 flags, struct netlink_ext_ack *extack);
+	/* This should be merged with the original init action */
+	int     (*init_ops)(struct net *net, struct nlattr *nla,
+			    struct nlattr *est, struct tc_action **act,
+			   struct tcf_proto *tp, struct tc_action_ops *ops,
+			   u32 flags, struct netlink_ext_ack *extack);
 	int     (*walk)(struct net *, struct sk_buff *,
 			struct netlink_callback *, int,
 			const struct tc_action_ops *,
net/sched/act_api.c +11 -3
index 2ff61be8e..c6a783a71 100644
@@ -1044,7 +1044,7 @@ int tcf_register_action(struct tc_action_ops *act,
 	struct tc_action_ops *a;
 	int ret;
 
-	if (!act->act || !act->dump || !act->init)
+	if (!act->act || !act->dump || (!act->init && !act->init_ops))
 		return -EINVAL;
 
 	/* We have to register pernet ops before making the action ops visible,
@@ -1517,8 +1517,16 @@ struct tc_action *tcf_action_init_1(struct net *net, struct tcf_proto *tp,
 			}
 		}
 
-		err = a_o->init(net, tb[TCA_ACT_OPTIONS], est, &a, tp,
-				userflags.value | flags, extack);
+		/* When we arrive here we guarantee that a_o->init or
+		 * a_o->init_ops exist.
+		 */
+		if (a_o->init)
+			err = a_o->init(net, tb[TCA_ACT_OPTIONS], est, &a, tp,
+					userflags.value | flags, extack);
+		else
+			err = a_o->init_ops(net, tb[TCA_ACT_OPTIONS], est, &a,
+					    tp, a_o, userflags.value | flags,
+					    extack);
 	} else {
 		err = a_o->init(net, nla, est, &a, tp, userflags.value | flags,
 				extack);
Signature · 1 line
2.34.1
Reviewed-by on v10 3/15
#8 lore raw

Re: [PATCH v10 net-next 03/15] net/sched: act_api: Update tc_action_ops to account for P4 actions

Quoted 14 lines · Jamal Hadi Salim

The initialisation of P4TC action instances require access to a struct p4tc_act (which appears in later patches) to help us to retrieve information like the P4 action parameters etc. In order to retrieve struct p4tc_act we need the pipeline name or id and the action name or id. Also recall that P4TC action IDs are P4 and are net namespace specific and not global like standard tc actions.
The init callback from tc_action_ops parameters had no way of supplying us that information. To solve this issue, we decided to create a new tc_action_ops callback (init_ops), that provies us with the tc_action_ops struct which then provides us with the pipeline and action name. In addition we add a new refcount to struct tc_action_ops called dyn_ref, which accounts for how many action instances we have of a specific action.

Quoted 6 lines
author maintainer
#9 lore raw

PATCH v10 4/15 net/sched: act_api: add struct p4tc_action_ops as a parameter to lookup callback

In reply to Jamal Hadi Salim

For P4 actions, we require information from struct tc_action_ops, specifically the action kind, to find and locate the P4 action information for the lookup operation.

---
 include/net/act_api.h | 3 ++-
 net/sched/act_api.c   | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)
include/net/act_api.h +2 -1
index bfded7ec6..c60f3ccf2 100644
@@ -116,7 +116,8 @@ struct tc_action_ops {
 		       struct tcf_result *); /* called under RCU BH lock*/
 	int     (*dump)(struct sk_buff *, struct tc_action *, int, int);
 	void	(*cleanup)(struct tc_action *);
-	int     (*lookup)(struct net *net, struct tc_action **a, u32 index);
+	int     (*lookup)(struct net *net, const struct tc_action_ops *ops,
+			  struct tc_action **a, u32 index);
 	int     (*init)(struct net *net, struct nlattr *nla,
 			struct nlattr *est, struct tc_action **act,
 			struct tcf_proto *tp,
net/sched/act_api.c +1 -1
index c6a783a71..869a38570 100644
@@ -726,7 +726,7 @@ static int __tcf_idr_search(struct net *net,
 	struct tc_action_net *tn = net_generic(net, ops->net_id);
 
 	if (unlikely(ops->lookup))
-		return ops->lookup(net, a, index);
+		return ops->lookup(net, ops, a, index);
 
 	return tcf_idr_search(tn, a, index);
 }
Signature · 1 line
2.34.1
Reviewed-by on v10 4/15
#10 lore raw

Re: [PATCH v10 net-next 04/15] net/sched: act_api: add struct p4tc_action_ops as a parameter to lookup callback

Quoted 8 lines · Jamal Hadi Salim

For P4 actions, we require information from struct tc_action_ops, specifically the action kind, to find and locate the P4 action information for the lookup operation.

author maintainer
#11 lore raw

PATCH v10 5/15 net: sched: act_api: Add support for preallocated P4 action instances

In reply to Jamal Hadi Salim

In P4, actions are assumed to pre exist and have an upper bound number of instances. Typically if you a table defined with 1M table entries you want to allocate enough action instances to cover the 1M entries. However, this is a big waste of memory if the action instances are not in use. So for our case, we allow the user to specify a minimal amount of actions in the template and then if more P4 action instances are needed then they will be added on demand as in the current approach with tc filter-action relationship.

Add the necessary code to preallocate actions instances for P4 actions.

We add 2 new actions flags:
- TCA_ACT_FLAGS_PREALLOC: Indicates the action instance is a P4 action and was preallocated for future use the templating phase of P4TC
- TCA_ACT_FLAGS_UNREFERENCED: Indicates the action instance was preallocated and is currently not being referenced by any other object. Which means it won't show up in an action instance dump.

Once an action instance is created we don't free it when the last table entry referring to it is deleted.
Instead we add it to the pool/cache of action instances for that specific action kind i.e it counts as if it is preallocated.
Preallocated actions can't be deleted by the tc actions runtime commands and a dump or a get will only show preallocated actions instances which are being used (i.e TCA_ACT_FLAGS_UNREFERENCED == false).

The preallocated actions will be deleted once the pipeline is deleted (which will purge the P4 action kind and its instances).

For example, if we were to create a P4 action that preallocates 128 elements and dumped:

$ tc -j p4template get action/myprog/send_nh | jq .

We'd see the following:

[

  {
    "obj": "action template",
    "pname": "myprog",
    "pipeid": 1
  },
  {
    "templates": [
      {
        "aname": "myprog/send_nh",
        "actid": 1,
        "params": [
          {
            "name": "port",
            "type": "dev",
            "id": 1
          }
        ],
        "prealloc": 128
      }
    ]

}
]

If we try to dump the P4 action instances, we won't see any:

$ tc -j actions ls action myprog/send_nh | jq .

[]

However, if we create a table entry which references this action kind:

$ tc p4ctrl create myprog/table/cb/FDB \
   dstAddr d2:96:91:5d:02:86 action myprog/send_nh \
   param port type dev dummy0

Dumping the action instance will now show this one instance which is associated with the table entry:

$ tc -j actions ls action myprog/send_nh | jq .

[

  {
    "total acts": 1
  },
  {
    "actions": [
      {
        "order": 0,
        "kind": "myprog/send_nh",
        "index": 1,
        "ref": 1,
        "bind": 1,
        "params": [
          {
            "name": "port",
            "type": "dev",
            "value": "dummy0",
            "id": 1
          }
        ],
        "not_in_hw": true
      }
    ]

}
]

---
 include/net/act_api.h |  3 +++
 net/sched/act_api.c   | 45 +++++++++++++++++++++++++++++++++++--------
 2 files changed, 40 insertions(+), 8 deletions(-)
include/net/act_api.h +3 -0
index c60f3ccf2..4b6a5bdca 100644
@@ -68,6 +68,8 @@ struct tc_action {
 #define TCA_ACT_FLAGS_REPLACE	(1U << (TCA_ACT_FLAGS_USER_BITS + 2))
 #define TCA_ACT_FLAGS_NO_RTNL	(1U << (TCA_ACT_FLAGS_USER_BITS + 3))
 #define TCA_ACT_FLAGS_AT_INGRESS	(1U << (TCA_ACT_FLAGS_USER_BITS + 4))
+#define TCA_ACT_FLAGS_PREALLOC	(1U << (TCA_ACT_FLAGS_USER_BITS + 5))
+#define TCA_ACT_FLAGS_UNREFERENCED	(1U << (TCA_ACT_FLAGS_USER_BITS + 6))
 
 /* Update lastuse only if needed, to avoid dirtying a cache line.
  * We use a temp variable to avoid fetching jiffies twice.
@@ -201,6 +203,7 @@ int tcf_idr_create_from_flags(struct tc_action_net *tn, u32 index,
 			      const struct tc_action_ops *ops, int bind,
 			      u32 flags);
 void tcf_idr_insert_many(struct tc_action *actions[], int init_res[]);
+void tcf_idr_insert_n(struct tc_action *actions[], const u32 n);
 void tcf_idr_cleanup(struct tc_action_net *tn, u32 index);
 int tcf_idr_check_alloc(struct tc_action_net *tn, u32 *index,
 			struct tc_action **a, int bind);
net/sched/act_api.c +37 -8

82 lines not on this page · raw message

Signature · 1 line
2.34.1
#12 lore raw

Re: [PATCH v10 net-next 05/15] net: sched: act_api: Add support for preallocated P4 action instances

Quoted 5 lines · Jamal Hadi Salim
$ tc -j actions ls action myprog/send_nh | jq .

[

  {
    "total acts": 1

...

Quoted 1 line
        "not_in_hw": true

For a moment I was like "hmm, this is going to get tricky. Some times space, sometimes _", but this is not introduced by this patch.

author maintainer
#13 lore raw

Re: [PATCH v10 net-next 05/15] net: sched: act_api: Add support for preallocated P4 action instances

Quoted 4 lines · Marcelo Ricardo Leitner
Quoted 2 lines · Jamal Hadi Salim
$ tc -j actions ls action myprog/send_nh | jq .
Quoted 8 lines
Quoted 3 lines

[

  {
    "total acts": 1

...

Quoted 1 line
        "not_in_hw": true

For a moment I was like "hmm, this is going to get tricky. Some times space, sometimes _", but this is not introduced by this patch.

Yes, unfortunately that is baked in into the actions code in iproute2. Would have been nice to say
"in h/w": false

cheers,
jamal

Quoted 3 lines
author maintainer
#14 lore raw

PATCH v10 6/15 p4tc: add P4 data types

In reply to Jamal Hadi Salim

Introduce abstraction that represents P4 data types.
This also introduces the Kconfig and Makefile which later patches use. Numeric types could be little, host or big endian definitions. The abstraction also supports defining:

a) bitstrings using P4 annotations that look like "bit<X>" where X is the number of bits defined in a type

b) bitslices such that one can define in P4 as bit<8>[0-3] and bit<16>[4-9]. A 4-bit slice from bits 0-3 and a 6-bit slice from bits 4-9 respectively.

c) speacialized types like dev (which stands for a netdev), key, etc

Each type has a bitsize, a name (for debugging purposes), an ID and methods/ops. The P4 types will be used by externs, dynamic actions, packet headers and other parts of P4TC.

Each type has four ops:

- validate_p4t: Which validates if a given value of a specific type meets valid boundary conditions.

- create_bitops: Which, given a bitsize, bitstart and bitend allocates and returns a mask and a shift value. For example, if we have type bit<8>[3-3] meaning bitstart = 3 and bitend = 3, we'll create a mask which would only give us the fourth bit of a bit8 value, that is, 0x08. Since we are interested in the fourth bit, the bit shift value will be 3. This is also useful if an "irregular" bitsize is used, for example, bit24. In that case bitstart = 0 and bitend = 23. Shift will be 0 and the mask will be 0xFFFFFF00 if the machine is big endian.

- host_read : Which reads the value of a given type and transforms it to host order (if needed)

- host_write : Which writes a provided host order value and transforms it to the type's native order (if needed)

---
 include/net/p4tc_types.h    |   91 +++
 include/uapi/linux/p4tc.h   |   33 +
 net/sched/Kconfig           |   11 +
 net/sched/Makefile          |    2 +
 net/sched/p4tc/Makefile     |    3 +
 net/sched/p4tc/p4tc_types.c | 1287 +++++++++++++++++++++++++++++++++++
 6 files changed, 1427 insertions(+)
 create mode 100644 include/net/p4tc_types.h
 create mode 100644 include/uapi/linux/p4tc.h
 create mode 100644 net/sched/p4tc/Makefile
 create mode 100644 net/sched/p4tc/p4tc_types.c
include/net/p4tc_types.h new file +91 -0
new file mode 100644
index 000000000..af9f51fc1
@@ -0,0 +1,91 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __NET_P4TYPES_H
+#define __NET_P4TYPES_H
+
+#include <linux/netlink.h>
+#include <linux/pkt_cls.h>
+#include <linux/types.h>
+
+#include <uapi/linux/p4tc.h>
+
+#define P4TC_T_MAX_BITSZ 128
+
+struct p4tc_type_mask_shift {
+	void *mask;
+	u8 shift;
+};
+
+struct p4tc_type;
+struct p4tc_type_ops {
+	int (*validate_p4t)(struct p4tc_type *container, void *value,
+			    u16 startbit, u16 endbit,
+			    struct netlink_ext_ack *extack);
+	struct p4tc_type_mask_shift *(*create_bitops)(u16 bitsz, u16 bitstart,
+						      u16 bitend,
+						      struct netlink_ext_ack *extack);
+	void (*host_read)(struct p4tc_type *container,
+			  struct p4tc_type_mask_shift *mask_shift, void *sval,
+			  void *dval);
+	void (*host_write)(struct p4tc_type *container,
+			   struct p4tc_type_mask_shift *mask_shift, void *sval,
+			   void *dval);
+	void (*print)(struct net *net, struct p4tc_type *container,
+		      const char *prefix, void *val);
+};
+
+#define P4TC_T_MAX_STR_SZ 32
+struct p4tc_type {
+	char name[P4TC_T_MAX_STR_SZ];
+	const struct p4tc_type_ops *ops;
+	size_t container_bitsz;
+	size_t bitsz;
+	int typeid;
+};
+
+struct p4tc_type *p4type_find_byid(int id);
+bool p4tc_is_type_unsigned_he(int typeid);
+bool p4tc_is_type_numeric(int typeid);
+
+void p4t_copy(struct p4tc_type_mask_shift *dst_mask_shift,
+	      struct p4tc_type *dst_t, void *dstv,
+	      struct p4tc_type_mask_shift *src_mask_shift,
+	      struct p4tc_type *src_t, void *srcv);
+int p4t_cmp(struct p4tc_type_mask_shift *dst_mask_shift,
+	    struct p4tc_type *dst_t, void *dstv,
+	    struct p4tc_type_mask_shift *src_mask_shift,
+	    struct p4tc_type *src_t, void *srcv);
+void p4t_release(struct p4tc_type_mask_shift *mask_shift);
+
+int p4tc_register_types(void);
+void p4tc_unregister_types(void);
+
+#ifdef CONFIG_RETPOLINE
+void __p4tc_type_host_read(const struct p4tc_type_ops *ops,
+			   struct p4tc_type *container,
+			   struct p4tc_type_mask_shift *mask_shift, void *sval,
+			   void *dval);
+void __p4tc_type_host_write(const struct p4tc_type_ops *ops,
+			    struct p4tc_type *container,
+			    struct p4tc_type_mask_shift *mask_shift, void *sval,
+			    void *dval);
+#else
+static inline void
+__p4tc_type_host_read(const struct p4tc_type_ops *ops,
+		      struct p4tc_type *container,
+		      struct p4tc_type_mask_shift *mask_shift,
+		      void *sval, void *dval)
+{
+	return ops->host_read(container, mask_shift, sval, dval);
+}
+
+static inline void
+__p4tc_type_host_write(const struct p4tc_type_ops *ops,
+		       struct p4tc_type *container,
+		       struct p4tc_type_mask_shift *mask_shift,
+		       void *sval, void *dval)
+{
+	return ops->host_write(container, mask_shift, sval, dval);
+}
+#endif
+
+#endif
include/uapi/linux/p4tc.h new file +33 -0
new file mode 100644
index 000000000..0133947c5
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __LINUX_P4TC_H
+#define __LINUX_P4TC_H
+
+#define P4TC_MAX_KEYSZ 512
+
+enum {
+	P4TC_T_UNSPEC,
+	P4TC_T_U8,
+	P4TC_T_U16,
+	P4TC_T_U32,
+	P4TC_T_U64,
+	P4TC_T_STRING,
+	P4TC_T_S8,
+	P4TC_T_S16,
+	P4TC_T_S32,
+	P4TC_T_S64,
+	P4TC_T_MACADDR,
+	P4TC_T_IPV4ADDR,
+	P4TC_T_BE16,
+	P4TC_T_BE32,
+	P4TC_T_BE64,
+	P4TC_T_U128,
+	P4TC_T_S128,
+	P4TC_T_BOOL,
+	P4TC_T_DEV,
+	P4TC_T_KEY,
+	__P4TC_T_MAX,
+};
+
+#define P4TC_T_MAX (__P4TC_T_MAX - 1)
+
+#endif
net/sched/Kconfig +11 -0
index 470c70def..b1798a5e1 100644
@@ -675,6 +675,17 @@ config NET_EMATCH_IPT
 	  To compile this code as a module, choose M here: the
 	  module will be called em_ipt.
 
+config NET_P4TC
+	bool "P4TC support"
+	select NET_CLS_ACT
+	help
+	  Say Y here if you want to use P4 features on top of TC.
+	  P4 is an open source,  domain-specific programming language for
+	  specifying data plane behavior. By enabling P4TC you will be able to
+	  write a P4 program, use a P4 compiler that supports P4TC backend to
+	  generate all needed artificats, which when loaded allow you to
+	  introduce a new kernel datapath that can be controlled via TC.
+
 config NET_CLS_ACT
 	bool "Actions"
 	select NET_CLS
net/sched/Makefile +2 -0
index 82c3f78ca..581f9dd69 100644
@@ -81,3 +81,5 @@ obj-$(CONFIG_NET_EMATCH_TEXT)	+= em_text.o
 obj-$(CONFIG_NET_EMATCH_CANID)	+= em_canid.o
 obj-$(CONFIG_NET_EMATCH_IPSET)	+= em_ipset.o
 obj-$(CONFIG_NET_EMATCH_IPT)	+= em_ipt.o
+
+obj-$(CONFIG_NET_P4TC)		+= p4tc/
net/sched/p4tc/Makefile new file +3 -0
new file mode 100644
index 000000000..dd1358c9e
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-y := p4tc_types.o
net/sched/p4tc/p4tc_types.c new file +1287 -0

1,287 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v10 6/15
#15 lore raw

Re: [PATCH v10 net-next 06/15] p4tc: add P4 data types

Quoted 35 lines · Jamal Hadi Salim

Introduce abstraction that represents P4 data types.
This also introduces the Kconfig and Makefile which later patches use. Numeric types could be little, host or big endian definitions. The abstraction also supports defining:

a) bitstrings using P4 annotations that look like "bit<X>" where X is the number of bits defined in a type

b) bitslices such that one can define in P4 as bit<8>[0-3] and bit<16>[4-9]. A 4-bit slice from bits 0-3 and a 6-bit slice from bits 4-9 respectively.

c) speacialized types like dev (which stands for a netdev), key, etc

Each type has a bitsize, a name (for debugging purposes), an ID and methods/ops. The P4 types will be used by externs, dynamic actions, packet headers and other parts of P4TC.

Each type has four ops:

- validate_p4t: Which validates if a given value of a specific type meets valid boundary conditions.

- create_bitops: Which, given a bitsize, bitstart and bitend allocates and returns a mask and a shift value. For example, if we have type bit<8>[3-3] meaning bitstart = 3 and bitend = 3, we'll create a mask which would only give us the fourth bit of a bit8 value, that is, 0x08. Since we are interested in the fourth bit, the bit shift value will be 3. This is also useful if an "irregular" bitsize is used, for example, bit24. In that case bitstart = 0 and bitend = 23. Shift will be 0 and the mask will be 0xFFFFFF00 if the machine is big endian.

- host_read : Which reads the value of a given type and transforms it to host order (if needed)

Quoted 8 lines

- host_write : Which writes a provided host order value and transforms it to the type's native order (if needed)

author maintainer
#16 lore raw

PATCH v10 7/15 p4tc: add template API

In reply to Jamal Hadi Salim

Add p4tc template API that will serve as infrastructure for all future template objects (in this set pipeline, table, action, and more later)

This commit is not functional by itself. It needs the subsequent patch to be of any use. This commit's purpose is to ease review. If something were to break and you do git bisect to this patch it will not be helpful. In the next release we are planning to merge the two back again.

The template API infrastructure follows the CRUD (Create, Read/get, Update, and Delete) commands.

To issue a p4template create command the user will follow the below grammar:

tc p4template create objtype/[objpath] [objid] objparams

To show a more concrete example, to create a new pipeline (pipelines come in the next commit), the user would issue the following command:

tc p4template create pipeline/aP4proggie pipeid 1 numtables 1 ...

Note that the user may specify an optional ID to the obj, if none is specified, the kernel will assign one.

The command for update is analogous:

tc p4template update objtype/[objpath] [objid] objparams

Note that for the user may refer to the object by name (in the objpath) or directly by ID.

Delete is also analogous:

tc p4template delete objtype/[objpath] [objid]

As is get:

tc p4template get objtype/[objpath] [objid]

One can also dump or flush template objects. This will be better exposed in the object specific commits in this patchset

---
 include/net/p4tc.h             |  53 ++++
 include/uapi/linux/p4tc.h      |  42 +++
 include/uapi/linux/rtnetlink.h |   9 +
 net/sched/p4tc/Makefile        |   2 +-
 net/sched/p4tc/p4tc_tmpl_api.c | 499 +++++++++++++++++++++++++++++++++
 security/selinux/nlmsgtab.c    |   6 +-
 6 files changed, 609 insertions(+), 2 deletions(-)
 create mode 100644 include/net/p4tc.h
 create mode 100644 net/sched/p4tc/p4tc_tmpl_api.c
include/net/p4tc.h new file +53 -0
new file mode 100644
index 000000000..c99862fe6
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __NET_P4TC_H
+#define __NET_P4TC_H
+
+#include <uapi/linux/p4tc.h>
+#include <linux/workqueue.h>
+#include <net/sch_generic.h>
+#include <net/net_namespace.h>
+#include <linux/refcount.h>
+#include <linux/rhashtable.h>
+#include <linux/rhashtable-types.h>
+
+#define P4TC_PATH_MAX 3
+
+struct p4tc_dump_ctx {
+	u32 ids[P4TC_PATH_MAX];
+};
+
+struct p4tc_template_common;
+
+struct p4tc_template_ops {
+	void (*init)(void);
+	struct p4tc_template_common *(*cu)(struct net *net, struct nlmsghdr *n,
+					   struct nlattr *nla,
+					   struct netlink_ext_ack *extack);
+	int (*put)(struct p4tc_template_common *tmpl,
+		   struct netlink_ext_ack *extack);
+	int (*gd)(struct net *net, struct sk_buff *skb, struct nlmsghdr *n,
+		  struct nlattr *nla, struct netlink_ext_ack *extack);
+	int (*fill_nlmsg)(struct net *net, struct sk_buff *skb,
+			  struct p4tc_template_common *tmpl,
+			  struct netlink_ext_ack *extack);
+	int (*dump)(struct sk_buff *skb, struct p4tc_dump_ctx *ctx,
+		    struct nlattr *nla, u32 *ids,
+		    struct netlink_ext_ack *extack);
+	int (*dump_1)(struct sk_buff *skb, struct p4tc_template_common *common);
+};
+
+struct p4tc_template_common {
+	char                     name[P4TC_TMPL_NAMSZ];
+	struct p4tc_template_ops *ops;
+};
+
+static inline bool p4tc_tmpl_msg_is_update(struct nlmsghdr *n)
+{
+	return n->nlmsg_type == RTM_UPDATEP4TEMPLATE;
+}
+
+int p4tc_tmpl_generic_dump(struct sk_buff *skb, struct p4tc_dump_ctx *ctx,
+			   struct idr *idr, int idx,
+			   struct netlink_ext_ack *extack);
+
+#endif
include/uapi/linux/p4tc.h +42 -0

54 lines not on this page · raw message

include/uapi/linux/rtnetlink.h +9 -0
index 3b687d20c..4f9ebe3e7 100644
@@ -194,6 +194,15 @@ enum {
 	RTM_GETTUNNEL,
 #define RTM_GETTUNNEL	RTM_GETTUNNEL
 
+	RTM_CREATEP4TEMPLATE = 124,
+#define RTM_CREATEP4TEMPLATE	RTM_CREATEP4TEMPLATE
+	RTM_DELP4TEMPLATE,
+#define RTM_DELP4TEMPLATE	RTM_DELP4TEMPLATE
+	RTM_GETP4TEMPLATE,
+#define RTM_GETP4TEMPLATE	RTM_GETP4TEMPLATE
+	RTM_UPDATEP4TEMPLATE,
+#define RTM_UPDATEP4TEMPLATE	RTM_UPDATEP4TEMPLATE
+
 	__RTM_MAX,
 #define RTM_MAX		(((__RTM_MAX + 3) & ~3) - 1)
 };
net/sched/p4tc/Makefile +1 -1
index dd1358c9e..e28dfc6eb 100644
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 
-obj-y := p4tc_types.o
+obj-y := p4tc_types.o p4tc_tmpl_api.o
net/sched/p4tc/p4tc_tmpl_api.c new file +499 -0

499 lines not on this page · raw message

security/selinux/nlmsgtab.c +5 -1
index 8ff670cf1..e50a1c1ff 100644
@@ -94,6 +94,10 @@ static const struct nlmsg_perm nlmsg_route_perms[] = {
 	{ RTM_NEWTUNNEL,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
 	{ RTM_DELTUNNEL,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
 	{ RTM_GETTUNNEL,	NETLINK_ROUTE_SOCKET__NLMSG_READ  },
+	{ RTM_CREATEP4TEMPLATE,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
+	{ RTM_DELP4TEMPLATE,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
+	{ RTM_GETP4TEMPLATE,	NETLINK_ROUTE_SOCKET__NLMSG_READ },
+	{ RTM_UPDATEP4TEMPLATE,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
 };
 
 static const struct nlmsg_perm nlmsg_tcpdiag_perms[] = {
@@ -177,7 +181,7 @@ int selinux_nlmsg_lookup(u16 sclass, u16 nlmsg_type, u32 *perm)
 		 * structures at the top of this file with the new mappings
 		 * before updating the BUILD_BUG_ON() macro!
 		 */
-		BUILD_BUG_ON(RTM_MAX != (RTM_NEWTUNNEL + 3));
+		BUILD_BUG_ON(RTM_MAX != (RTM_CREATEP4TEMPLATE + 3));
 		err = nlmsg_perm(nlmsg_type, perm, nlmsg_route_perms,
 				 sizeof(nlmsg_route_perms));
 		break;
Signature · 1 line
2.34.1
author maintainer
#18 lore raw

PATCH v10 8/15 p4tc: add template pipeline create, get, update, delete

In reply to Jamal Hadi Salim

__Introducing P4 TC Pipeline__

This commit introduces P4 TC pipelines, which emulate the semantics of a P4 program/pipeline using the TC infrastructure.

This patch relies on the previous one to be functional. They were split to ease review. In the next release we are planning this and the previous patch back again.

One can refer to P4 programs/pipelines using their names or their specific pipeline ids (pipeid)

P4 template CRUD (Create, Read/get, Update and Delete) commands apply on a pipeline.

As an example, to create a P4 program/pipeline named aP4proggie with a single table in its pipeline, one would use the following command from user space tc (as generated by the compiler):

tc p4template create pipeline/aP4proggie numtables 1 pipeid 1

Note that, in the above command, the numtables is set as 1; the default is 0 because it is feasible to have a P4 program with no tables at all.

The P4 compiler will generate the pipeid, however if none is specified, the kernel will issue one. Like the following example:

tc p4template create pipeline/aP4proggie numtables 1

To Read pipeline aP4proggie attributes, one would retrieve those details as follows:

tc p4template get pipeline/[aP4proggie] [pipeid 1]

Note that in the above command one may specify pipeline ID, name or both.

To Update aP4proggie pipeline from 1 to 10 tables, one would use the following command:

tc p4template update pipeline/[aP4proggie] [pipeid 1] numtables 10

Note that, in the above command, one could use the P4 program/pipeline name, id or both to specify which P4 program/pipeline to update.

To Delete a P4 program/pipeline named aP4proggie with a pipeid of 1, one would use the following command:

tc p4template del pipeline/[aP4proggie] [pipeid 1]

Note that, in the above command, one could use the P4 program/pipeline name, id or both to specify which P4 program/pipeline to delete

If one wished to dump all the created P4 programs/pipelines, one would use the following command:

tc p4template get pipeline/

__Pipeline Lifetime__

After Create is issued, one can Read/get, Update and Delete; however the pipeline can only be put to use after it is "sealed". To seal a pipeline, one would issue the following command:

tc p4template update pipeline/aP4proggie state ready

After a pipeline is sealed it can be put to use via the TC P4 classifier. For example:

tc filter add $DEV ingress protocol any prio 6 p4 pname aP4proggie \
action bpf obj $PARSER.o section prog/tc-parser action bpf obj $PROGNAME.o section prog/tc-ingress

Instantiates aP4proggie in the ingress of $DEV. One could also attach it to a block of ports (example tc block 22) as such:

tc filter add block 22 ingress protocol all prio 6 p4 pname aP4proggie \
action bpf obj $PARSER.o section prog/tc-parser
action bpf obj $PROGNAME.o section prog/tc-ingress

We can, add a table entry after the pipeline is sealed (even before instantiating). Like, for example:

tc p4ctrl create aP4proggie/table/cb/aP4table \
      dstAddr 10.10.10.0/24 srcAddr 192.168.0.0/16 prio 16 \
      action drop

Once the pipeline is instantiated on a device or block it cannot be deleted. It becomes Read-only from the control plane/user space.
The pipeline can be deleted when there are no longer any users left by destroying all instances.

---
 include/net/p4tc.h             |  78 ++++-
 include/uapi/linux/p4tc.h      |  24 ++
 net/sched/p4tc/Makefile        |   2 +-
 net/sched/p4tc/p4tc_pipeline.c | 614 +++++++++++++++++++++++++++++++++
 net/sched/p4tc/p4tc_tmpl_api.c | 136 ++++++--
 5 files changed, 825 insertions(+), 29 deletions(-)
 create mode 100644 net/sched/p4tc/p4tc_pipeline.c
include/net/p4tc.h +75 -3
index c99862fe6..33526db51 100644
@@ -10,28 +10,44 @@
 #include <linux/rhashtable.h>
 #include <linux/rhashtable-types.h>
 
+#define P4TC_DEFAULT_NUM_TABLES P4TC_MINTABLES_COUNT
+#define P4TC_DEFAULT_MAX_RULES 1
 #define P4TC_PATH_MAX 3
 
+#define P4TC_KERNEL_PIPEID 0
+
+#define P4TC_PID_IDX 0
+
 struct p4tc_dump_ctx {
 	u32 ids[P4TC_PATH_MAX];
 };
 
 struct p4tc_template_common;
 
+struct p4tc_path_nlattrs {
+	char                     *pname;
+	u32                      *ids;
+	bool                     pname_passed;
+};
+
+struct p4tc_pipeline;
 struct p4tc_template_ops {
 	void (*init)(void);
 	struct p4tc_template_common *(*cu)(struct net *net, struct nlmsghdr *n,
 					   struct nlattr *nla,
+					   struct p4tc_path_nlattrs *nl_pname,
 					   struct netlink_ext_ack *extack);
-	int (*put)(struct p4tc_template_common *tmpl,
+	int (*put)(struct p4tc_pipeline *pipeline,
+		   struct p4tc_template_common *tmpl,
 		   struct netlink_ext_ack *extack);
 	int (*gd)(struct net *net, struct sk_buff *skb, struct nlmsghdr *n,
-		  struct nlattr *nla, struct netlink_ext_ack *extack);
+		  struct nlattr *nla, struct p4tc_path_nlattrs *nl_pname,
+		  struct netlink_ext_ack *extack);
 	int (*fill_nlmsg)(struct net *net, struct sk_buff *skb,
 			  struct p4tc_template_common *tmpl,
 			  struct netlink_ext_ack *extack);
 	int (*dump)(struct sk_buff *skb, struct p4tc_dump_ctx *ctx,
-		    struct nlattr *nla, u32 *ids,
+		    struct nlattr *nla, char **p_name, u32 *ids,
 		    struct netlink_ext_ack *extack);
 	int (*dump_1)(struct sk_buff *skb, struct p4tc_template_common *common);
 };
@@ -39,6 +55,27 @@ struct p4tc_template_ops {
 struct p4tc_template_common {
 	char                     name[P4TC_TMPL_NAMSZ];
 	struct p4tc_template_ops *ops;
+	u32                      p_id;
+	u32                      PAD0;
+};
+
+extern const struct p4tc_template_ops p4tc_pipeline_ops;
+
+struct p4tc_pipeline {
+	struct p4tc_template_common common;
+	struct rcu_head             rcu;
+	struct net                  *net;
+	/* Accounts for how many entities are referencing this pipeline.
+	 * As for now only P4 filters can refer to pipelines.
+	 */
+	refcount_t                  p_ctrl_ref;
+	u16                         num_tables;
+	u16                         curr_tables;
+	u8                          p_state;
+};
+
+struct p4tc_pipeline_net {
+	struct idr pipeline_idr;
 };
 
 static inline bool p4tc_tmpl_msg_is_update(struct nlmsghdr *n)
@@ -50,4 +87,39 @@ int p4tc_tmpl_generic_dump(struct sk_buff *skb, struct p4tc_dump_ctx *ctx,
 			   struct idr *idr, int idx,
 			   struct netlink_ext_ack *extack);
 
+struct p4tc_pipeline *p4tc_pipeline_find_byany(struct net *net,
+					       const char *p_name,
+					       const u32 pipeid,
+					       struct netlink_ext_ack *extack);
+struct p4tc_pipeline *p4tc_pipeline_find_byid(struct net *net,
+					      const u32 pipeid);
+struct p4tc_pipeline *
+p4tc_pipeline_find_get(struct net *net, const char *p_name,
+		       const u32 pipeid, struct netlink_ext_ack *extack);
+
+static inline bool p4tc_pipeline_get(struct p4tc_pipeline *pipeline)
+{
+	return refcount_inc_not_zero(&pipeline->p_ctrl_ref);
+}
+
+void p4tc_pipeline_put(struct p4tc_pipeline *pipeline);
+struct p4tc_pipeline *
+p4tc_pipeline_find_byany_unsealed(struct net *net, const char *p_name,
+				  const u32 pipeid,
+				  struct netlink_ext_ack *extack);
+
+static inline int p4tc_action_destroy(struct tc_action **acts)
+{
+	int ret = 0;
+
+	if (acts) {
+		ret = tcf_action_destroy(acts, TCA_ACT_UNBIND);
+		kfree(acts);
+	}
+
+	return ret;
+}
+
+#define to_pipeline(t) ((struct p4tc_pipeline *)t)
+
 #endif
include/uapi/linux/p4tc.h +24 -0
index 22ba1c05a..8d8ffcb9e 100644
@@ -7,19 +7,25 @@
 
 /* pipeline header */
 struct p4tcmsg {
+	__u32 pipeid;
 	__u32 obj;
 };
 
+#define P4TC_MAXPIPELINE_COUNT 32
+#define P4TC_MAXTABLES_COUNT 32
+#define P4TC_MINTABLES_COUNT 0
 #define P4TC_MSGBATCH_SIZE 16
 
 #define P4TC_MAX_KEYSZ 512
 
 #define P4TC_TMPL_NAMSZ 32
+#define P4TC_PIPELINE_NAMSIZ P4TC_TMPL_NAMSZ
 
 /* Root attributes */
 enum {
 	P4TC_ROOT_UNSPEC,
 	P4TC_ROOT, /* nested messages */
+	P4TC_ROOT_PNAME, /* string - mandatory for pipeline create */
 	__P4TC_ROOT_MAX,
 };
 
@@ -28,6 +34,7 @@ enum {
 /* P4 Object types */
 enum {
 	P4TC_OBJ_UNSPEC,
+	P4TC_OBJ_PIPELINE,
 	__P4TC_OBJ_MAX,
 };
 
@@ -43,6 +50,23 @@ enum {
 
 #define P4TC_MAX (__P4TC_MAX - 1)
 
+/* PIPELINE attributes */
+enum {
+	P4TC_PIPELINE_UNSPEC,
+	P4TC_PIPELINE_NUMTABLES, /* u16 */
+	P4TC_PIPELINE_STATE, /* u8 */
+	P4TC_PIPELINE_NAME, /* string only used for pipeline dump */
+	__P4TC_PIPELINE_MAX
+};
+
+#define P4TC_PIPELINE_MAX (__P4TC_PIPELINE_MAX - 1)
+
+/* PIPELINE states */
+enum {
+	P4TC_STATE_NOT_READY,
+	P4TC_STATE_READY,
+};
+
 enum {
 	P4TC_T_UNSPEC,
 	P4TC_T_U8,
net/sched/p4tc/Makefile +1 -1
index e28dfc6eb..0881a7563 100644
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 
-obj-y := p4tc_types.o p4tc_tmpl_api.o
+obj-y := p4tc_types.o p4tc_tmpl_api.o p4tc_pipeline.o
net/sched/p4tc/p4tc_pipeline.c new file +614 -0

614 lines not on this page · raw message

net/sched/p4tc/p4tc_tmpl_api.c +111 -25

347 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v10 8/15
#19 lore raw

Re: [PATCH v10 net-next 08/15] p4tc: add template pipeline create, get, update, delete

include/net/p4tc.h struct p4tc_template_ops { · line 59

Quoted 6 lines · Jamal Hadi Salim
include/net/p4tc.h +2 -0
@@ -39,6 +55,27 @@ struct p4tc_template_ops {
 struct p4tc_template_common {
 	char                     name[P4TC_TMPL_NAMSZ];
 	struct p4tc_template_ops *ops;
+	u32                      p_id;
+	u32                      PAD0;

Perhaps __pad0 is more common. But, is it really needed?

Review struct p4tc_template_ops { · line 60

Quoted 1 line
(quoted diff) +1 -0
+};

Only nit.

author maintainer
#20 lore raw

Re: [PATCH v10 net-next 08/15] p4tc: add template pipeline create, get, update, delete

Quoted 11 lines · Marcelo Ricardo Leitner

include/net/p4tc.h struct p4tc_template_ops { · line 59

Quoted 6 lines · Jamal Hadi Salim
include/net/p4tc.h +2 -0
@@ -39,6 +55,27 @@ struct p4tc_template_ops {
 struct p4tc_template_common {
      char                     name[P4TC_TMPL_NAMSZ];
      struct p4tc_template_ops *ops;
+     u32                      p_id;
+     u32                      PAD0;

Perhaps __pad0 is more common. But, is it really needed?

$ pahole -C p4tc_template_common net/sched/p4tc/p4tc_tmpl_api.o
struct p4tc_template_common {
        char                       name[32];             /*     0    32 */
        struct p4tc_template_ops * ops;                  /*    32     8 */
        u32                        p_id;                 /*    40     4 */
        u32                        PAD0;                 /*    44     4 */
        /* size: 48, cachelines: 1, members: 4 */
        /* last cacheline: 48 bytes */
};

Looks good for 64b alignment. We can change the name.

Quoted 5 lines

Review struct p4tc_template_ops { · line 60

Quoted 1 line
(quoted diff) +1 -0
+};

Only nit.

Thanks for this and all the other reviews. Much appreciated!

cheers,
jamal

A message that is not in the archive (sent off-list or not yet received): CA+JHD91MqtsyrBD=OheYio7bddrqKGtXjgiC7oJDGQjpFU17iQ@mail.gmail.com

author maintainer
#22 lore raw

Re: [PATCH v10 net-next 08/15] p4tc: add template pipeline create, get, update, delete

Quoted 26 lines · Arnaldo Carvalho de Melo
Quoted 22 lines · Jamal Hadi Salim
Quoted 11 lines · Marcelo Ricardo Leitner

include/net/p4tc.h struct p4tc_template_ops { · line 59

Quoted 6 lines · Jamal Hadi Salim
include/net/p4tc.h +2 -0
@@ -39,6 +55,27 @@ struct p4tc_template_ops {
 struct p4tc_template_common {
      char                     name[P4TC_TMPL_NAMSZ];
      struct p4tc_template_ops *ops;
+     u32                      p_id;
+     u32                      PAD0;

Perhaps __pad0 is more common. But, is it really needed?

$ pahole -C p4tc_template_common net/sched/p4tc/p4tc_tmpl_api.o
struct p4tc_template_common {
        char                       name[32];             /*     0    32 */
        struct p4tc_template_ops * ops;                  /*    32     8 */
        u32                        p_id;                 /*    40     4 */
        u32                        PAD0;                 /*    44     4 */
Quoted 9 lines
Quoted 5 lines
        /* size: 48, cachelines: 1, members: 4 */
        /* last cacheline: 48 bytes */
};

Looks good for 64b alignment. We can change the name.

I bet that is you just remove PAD0 the compiler will introduce our for you.

True dat.

Quoted 1 line

Doing it explicitly documents explicitly tho.

Documentation justifies it - so we'll leave it there.

cheers,
jamal

Quoted 14 lines

- Arnaldo

Quoted 13 lines
Quoted 7 lines

Review struct p4tc_template_ops { · line 60

Quoted 3 lines
(quoted diff) +1 -0
+};

Only nit.

Thanks for this and all the other reviews. Much appreciated!

cheers,
jamal

author maintainer
#23 lore raw

PATCH v10 9/15 p4tc: add template action create, update, delete, get, flush and dump

In reply to Jamal Hadi Salim

This commit allows the control plane to create, update, delete, get, flush and dump action templates based on P4 action definitions.

Visualize the following action in a P4 program named aP4Proggie:

action send_nh(@tc_type("macaddr) bit<48> dstAddr, @tc_type("dev") bit<8> port)

{
    hdr.ethernet.dstAddr = dstMac;
    send_to_port(port);
}

The above is an action called send_nh which receives as parameters a bit<48> dstAddr (a mac address) and a bit<8> port (something close to ifindex).

which is applied on a P4 table match as such:

table mytable {
        key = {
            hdr.ipv4.dstAddr @tc_type("ipv4"): lpm;
        }

        actions = {
            send_nh;
            drop;
            NoAction;
        }

        size = 1024;
}

The mechanics of actions follow the CRUD semantics which are illustrated in the text below.

___P4 ACTION KIND CREATION___

In this stage we create the p4 action kind by specifying the action name, its ID, its parameters and the parameter types.
So for the send_nh action, the creation would look something like this:

tc p4template create action/aP4proggie/send_nh \
param dstAddr type macaddr id 1 param port type dev id 2

All the template commands (tc p4template) are generated by the p4c compiler (but of course could be hand coded by humans).

Also note that an action name has to specify the program name since P4 actions are unique to a program. As an example, the above command creates an action template that is bounded to pipeline/program named "aP4proggie".

Note2: In P4, actions are assumed to pre-exist and have an upper bound number of instances. Typically, if you have a max of 1024 "mytable" table entries you want to allocate enough action instances to cover the 1024 entries. However, this is a big waste of memory when we have low table occupancy. We pick a middle ground by providing pre-allocation control via attribute "num_prealloc".
The compiler generated template does not specify it and by default we preallocate 16 entries. The user can override this value by editing the generated text, for example to change the number to 128 as such:

tc p4template create action/aP4proggie/send_nh num_prealloc 128 \
param dstAddr type macaddr id 1 param port type dev id 2

When all preallocated action instances are exhausted (used in table entries) then the behavior switches to the current tc action approach i.e for every table entry created a new action instance is dynamically allocated if no "index" attribute is specified. Once an instance is created it is added to the pool and never freed.

Note, Current tc action behavior is maintained:

a) If the user wishes to preallocate more actions instance later at runtime to take advantage of a faster table entry creation (by avoiding dynamic allocation at table entry creation time), they will have to individually create actions via the control plane using the classical "tc actions" command.
For example:

tc actions add action aP4proggie/send_nh \
param dstAddr AA:BB:CC:DD:EE:DD param port eth1

The action is added to the pool of action aP4proggie/send_nh instances and any table entry creation will grab it. The parameters specified above will be replaced when the table entry is created.

b) Sharing of action instances works the same way (i.e you could autobind to any action instance in a table entry creation by specifying the action "index".

___ACTION KIND ACTIVATION___

Once we provided all the necessary information for the new p4 action, we can go to the final stage: action activation. In this template stage, we activate the p4 action and make it available for instantiation. To activate the action template, we issue the following command:

tc p4template update action/aP4proggie/send_nh state active

___OTHER CONTROL COMMANDS___

The lifetime of the p4 action is tied to its pipeline
(see earlier patches). As with all pipeline components, write operations to action templates (i.e action kinds), such as create, update and delete, can only be executed if the pipeline is not sealed. Read/get can be issued even after the pipeline is sealed.

If, after we are done with our action template we want to delete it, we could issue the following command:

tc p4template del action/aP4proggie/send_nh

Note: If any instance was created for this action (as illustrated earlier) than this action cannot be deleted, unless you delete all instances first.

If we had created more action templates and wanted to flush all of the action templates from pipeline aP4proggie, one would use the following command:

tc p4template del action/aP4proggie/

After creating or updating a p4 actions, if one wishes to verify that the p4 action was created correctly, one would use the following command:

tc p4template get action/aP4proggie/send_nh

The above command will display the relevant data for the action, such as parameter names, types, etc.

If one wanted to check which action templates were associated to a specific pipeline, one could use the following command:

tc p4template get action/aP4proggie/

Note that this command will only display the name of these action templates. To verify their specific details, one should use the get command, which was previously described.

---
 include/net/act_api.h             |    1 +
 include/net/p4tc.h                |   79 ++-
 include/net/tc_act/p4tc.h         |   28 +
 include/uapi/linux/p4tc.h         |   42 ++
 include/uapi/linux/tc_act/tc_p4.h |   11 +
 net/sched/p4tc/Makefile           |    3 +-
 net/sched/p4tc/p4tc_action.c      | 1080 +++++++++++++++++++++++++++++
 net/sched/p4tc/p4tc_pipeline.c    |   16 +-
 net/sched/p4tc/p4tc_tmpl_api.c    |   18 +
 9 files changed, 1267 insertions(+), 11 deletions(-)
 create mode 100644 include/net/tc_act/p4tc.h
 create mode 100644 include/uapi/linux/tc_act/tc_p4.h
 create mode 100644 net/sched/p4tc/p4tc_action.c
include/net/act_api.h +1 -0
index 4b6a5bdca..d7c01e397 100644
@@ -70,6 +70,7 @@ struct tc_action {
 #define TCA_ACT_FLAGS_AT_INGRESS	(1U << (TCA_ACT_FLAGS_USER_BITS + 4))
 #define TCA_ACT_FLAGS_PREALLOC	(1U << (TCA_ACT_FLAGS_USER_BITS + 5))
 #define TCA_ACT_FLAGS_UNREFERENCED	(1U << (TCA_ACT_FLAGS_USER_BITS + 6))
+#define TCA_ACT_FLAGS_FROM_P4TC	(1U << (TCA_ACT_FLAGS_USER_BITS + 7))
 
 /* Update lastuse only if needed, to avoid dirtying a cache line.
  * We use a temp variable to avoid fetching jiffies twice.
include/net/p4tc.h +71 -8

114 lines not on this page · raw message

include/net/tc_act/p4tc.h new file +28 -0
new file mode 100644
index 000000000..6447fe5ce
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __NET_TC_ACT_P4_H
+#define __NET_TC_ACT_P4_H
+
+#include <net/pkt_cls.h>
+#include <net/act_api.h>
+
+struct tcf_p4act_params {
+	struct tcf_exts exts;
+	struct idr params_idr;
+	struct p4tc_act_param **params_array;
+	struct rcu_head rcu;
+	u32 num_params;
+	u32 tot_params_sz;
+};
+
+struct tcf_p4act {
+	struct tc_action common;
+	/* Params IDR reference passed during runtime */
+	struct tcf_p4act_params __rcu *params;
+	u32 p_id;
+	u32 act_id;
+	struct list_head node;
+};
+
+#define to_p4act(a) ((struct tcf_p4act *)a)
+
+#endif /* __NET_TC_ACT_P4_H */
include/uapi/linux/p4tc.h +42 -0

75 lines not on this page · raw message

include/uapi/linux/tc_act/tc_p4.h new file +11 -0
new file mode 100644
index 000000000..874d85c9f
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __LINUX_TC_P4_H
+#define __LINUX_TC_P4_H
+
+#include <linux/pkt_cls.h>
+
+struct tc_act_p4 {
+	tc_gen;
+};
+
+#endif
net/sched/p4tc/Makefile +2 -1
index 0881a7563..7dbcf8915 100644
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
 
-obj-y := p4tc_types.o p4tc_tmpl_api.o p4tc_pipeline.o
+obj-y := p4tc_types.o p4tc_tmpl_api.o p4tc_pipeline.o \
+	p4tc_action.o
net/sched/p4tc/p4tc_action.c new file +1080 -0

1,080 lines not on this page · raw message

net/sched/p4tc/p4tc_pipeline.c +14 -2
index 6a4e7d5f7..beacdb874 100644
@@ -75,6 +75,8 @@ static const struct nla_policy tc_pipeline_policy[P4TC_PIPELINE_MAX + 1] = {
 
 static void p4tc_pipeline_destroy(struct p4tc_pipeline *pipeline)
 {
+	idr_destroy(&pipeline->p_act_idr);
+
 	kfree(pipeline);
 }
 
@@ -96,8 +98,12 @@ static void p4tc_pipeline_teardown(struct p4tc_pipeline *pipeline,
 	struct net *net = pipeline->net;
 	struct p4tc_pipeline_net *pipe_net = net_generic(net, pipeline_net_id);
 	struct net *pipeline_net = maybe_get_net(net);
+	unsigned long iter_act_id;
+	struct p4tc_act *act;
+	unsigned long tmp;
 
-	idr_remove(&pipe_net->pipeline_idr, pipeline->common.p_id);
+	idr_for_each_entry_ul(&pipeline->p_act_idr, act, tmp, iter_act_id)
+		act->common.ops->put(pipeline, &act->common, extack);
 
 	/* If we are on netns cleanup we can't touch the pipeline_idr.
 	 * On pre_exit we will destroy the idr but never call into teardown
@@ -154,6 +160,7 @@ static inline int pipeline_try_set_state_ready(struct p4tc_pipeline *pipeline,
 	}
 
 	pipeline->p_state = P4TC_STATE_READY;
+
 	return true;
 }
 
@@ -251,6 +258,10 @@ p4tc_pipeline_create(struct net *net, struct nlmsghdr *n,
 	else
 		pipeline->num_tables = P4TC_DEFAULT_NUM_TABLES;
 
+	idr_init(&pipeline->p_act_idr);
+
+	pipeline->num_created_acts = 0;
+
 	pipeline->p_state = P4TC_STATE_NOT_READY;
 
 	pipeline->net = net;
@@ -505,7 +516,8 @@ static int p4tc_pipeline_gd(struct net *net, struct sk_buff *skb,
 		return PTR_ERR(pipeline);
 
 	tmpl = (struct p4tc_template_common *)pipeline;
-	if (p4tc_pipeline_fill_nlmsg(net, skb, tmpl, extack) < 0)
+	ret = p4tc_pipeline_fill_nlmsg(net, skb, tmpl, extack);
+	if (ret < 0)
 		return -1;
 
 	if (!ids[P4TC_PID_IDX])
net/sched/p4tc/p4tc_tmpl_api.c +18 -0
index 1140c128a..7126ff7cd 100644
@@ -42,6 +42,7 @@ static bool obj_is_valid(u32 obj)
 {
 	switch (obj) {
 	case P4TC_OBJ_PIPELINE:
+	case P4TC_OBJ_ACT:
 		return true;
 	default:
 		return false;
@@ -50,6 +51,7 @@ static bool obj_is_valid(u32 obj)
 
 static const struct p4tc_template_ops *p4tc_ops[P4TC_OBJ_MAX + 1] = {
 	[P4TC_OBJ_PIPELINE] = &p4tc_pipeline_ops,
+	[P4TC_OBJ_ACT] = &p4tc_act_ops,
 };
 
 int p4tc_tmpl_generic_dump(struct sk_buff *skb, struct p4tc_dump_ctx *ctx,
@@ -124,6 +126,11 @@ static int tc_ctl_p4_tmpl_gd_1(struct net *net, struct sk_buff *skb,
 
 	ids[P4TC_PID_IDX] = t->pipeid;
 
+	if (tb[P4TC_PATH]) {
+		const u32 *arg_ids = nla_data(tb[P4TC_PATH]);
+
+		memcpy(&ids[P4TC_PID_IDX + 1], arg_ids, nla_len(tb[P4TC_PATH]));
+	}
 	nl_path_attrs->ids = ids;
 
 	obj_op = (struct p4tc_template_ops *)p4tc_ops[t->obj];
@@ -311,6 +318,12 @@ p4tc_tmpl_cu_1(struct sk_buff *skb, struct net *net, struct nlmsghdr *n,
 	}
 
 	ids[P4TC_PID_IDX] = t->pipeid;
+
+	if (tb[P4TC_PATH]) {
+		const u32 *arg_ids = nla_data(tb[P4TC_PATH]);
+
+		memcpy(&ids[P4TC_PID_IDX + 1], arg_ids, nla_len(tb[P4TC_PATH]));
+	}
 	nl_path_attrs->ids = ids;
 
 	obj_op = (struct p4tc_template_ops *)p4tc_ops[t->obj];
@@ -504,6 +517,11 @@ static int tc_ctl_p4_tmpl_dump_1(struct sk_buff *skb, struct nlattr *arg,
 	root = nla_nest_start(skb, P4TC_ROOT);
 
 	ids[P4TC_PID_IDX] = t->pipeid;
+	if (tb[P4TC_PATH]) {
+		const u32 *arg_ids = nla_data(tb[P4TC_PATH]);
+
+		memcpy(&ids[P4TC_PID_IDX + 1], arg_ids, nla_len(tb[P4TC_PATH]));
+	}
 
 	obj_op = (struct p4tc_template_ops *)p4tc_ops[t->obj];
 	ret = obj_op->dump(skb, ctx, tb[P4TC_PARAMS], &p_name, ids, extack);
Signature · 1 line
2.34.1
Reviewed-by on v10 9/15
#24 lore raw

Re: [PATCH v10 net-next 09/15] p4tc: add template action create, update, delete, get, flush and dump

author maintainer
#25 lore raw

PATCH v10 10/15 p4tc: add runtime action support

In reply to Jamal Hadi Salim

This commit deals with the runtime part of P4 actions i.e instantiation and binding of action kinds which are created via templates (see previous patch).

For illustration we repeat the P4 code snippet from the action template commit:

action send_nh(@tc_type("macaddr) bit<48> dstAddr, @tc_type("dev") bit<8> port)

{
    hdr.ethernet.dstAddr = dstMac;
    send_to_port(port);
}
table mytable {
        key = {
            hdr.ipv4.dstAddr @tc_type("ipv4"): lpm;
        }

        actions = {
            send_nh;
            drop;
            NoAction;
        }

        size = 1024;
}

One could create a table entry alongside an action instance as follows:

tc p4ctrl create aP4proggie/table/mycontrol/mytable \
srcAddr 10.10.10.0/24 \
action send_nh param dstAddr AA:BB:CC:DD:EE:FF param port eth0

As previously stated, we refer to the action by it's "full name" (pipeline_name/action_name). In the above the pipeline_name is inheritted from the table patch ("aP4proggie"), so it doesn't need to be specified. Above we are creating an instance of the send_nh action specifying as parameter values AA:BB:CC:DD:EE:FF for dstAddr and eth0 for port.

We could also create the action create an instance outside of the table entry which is then added to the pool of send_nh action kind's instances as follows:

tc actions add action aP4proggie/send_nh \
param dstAddr AA:BB:CC:DD:EE:FF param port eth0

Observe these are _exactly the same semantics_ as what tc today already provides with a caveat that we have a keyword "param" to precede the appropriate parameters.

Note: We can create as many instances for action templates as we wish, as long as we do not exceed the maximum allowed actions - in this specific case 1024 for table "mytable".

Action sharing still works the same way (as in classical tc). For example if we know the action index of the previous instance is 100 then we can bind it to a table entry, for example

tc p4ctrl create aP4proggie/table/mycontrol/mytable \
srcAddr 11.11.0.0/16 action send_nh index 100

---
 include/net/p4tc.h           |  102 +++
 include/uapi/linux/p4tc.h    |   13 +
 net/sched/p4tc/p4tc_action.c | 1179 +++++++++++++++++++++++++++++++++-
 3 files changed, 1283 insertions(+), 11 deletions(-)
include/net/p4tc.h +102 -0
index f77028e35..7b06c63cd 100644
@@ -116,6 +116,59 @@ p4tc_pipeline_find_byany_unsealed(struct net *net, const char *p_name,
 				  const u32 pipeid,
 				  struct netlink_ext_ack *extack);
 
+struct p4tc_act *p4a_runt_find(struct net *net,
+			       const struct tc_action_ops *a_o,
+			       struct netlink_ext_ack *extack);
+void
+p4a_runt_prealloc_put(struct p4tc_act *act, struct tcf_p4act *p4_act);
+
+static inline int p4tc_action_destroy(struct tc_action **acts)
+{
+	struct tc_action *acts_non_prealloc[TCA_ACT_MAX_PRIO] = {NULL};
+	int ret = 0;
+
+	if (acts) {
+		int j = 0;
+		int i;
+
+		for (i = 0; i < TCA_ACT_MAX_PRIO && acts[i]; i++) {
+			if (acts[i]->tcfa_flags & TCA_ACT_FLAGS_PREALLOC) {
+				struct tcf_p4act *p4act;
+				struct p4tc_act *act;
+				struct net *net;
+
+				p4act = (struct tcf_p4act *)acts[i];
+				net = maybe_get_net(acts[i]->idrinfo->net);
+
+				if (net) {
+					const struct tc_action_ops *ops;
+
+					ops = acts[i]->ops;
+					act = p4a_runt_find(net, ops, NULL);
+					p4a_runt_prealloc_put(act, p4act);
+					put_net(net);
+				} else {
+					/* If net is coming down, template
+					 * action will be deleted, so no need to
+					 * remove from prealloc list, just decr
+					 * refcounts.
+					 */
+					acts_non_prealloc[j] = acts[i];
+					j++;
+				}
+			} else {
+				acts_non_prealloc[j] = acts[i];
+				j++;
+			}
+		}
+
+		ret = tcf_action_destroy(acts_non_prealloc, TCA_ACT_UNBIND);
+		kfree(acts);
+	}
+
+	return ret;
+}
+
 struct p4tc_act_param {
 	struct list_head head;
 	struct rcu_head	rcu;
@@ -170,6 +223,47 @@ struct p4tc_act {
 
 extern const struct p4tc_template_ops p4tc_act_ops;
 
+static inline int p4tc_action_init(struct net *net, struct nlattr *nla,
+				   struct tc_action *acts[], u32 pipeid,
+				   u32 flags, struct netlink_ext_ack *extack)
+{
+	int init_res[TCA_ACT_MAX_PRIO];
+	size_t attrs_size;
+	int ret;
+	int i;
+
+	/* If action was already created, just bind to existing one */
+	flags |= TCA_ACT_FLAGS_BIND;
+	flags |= TCA_ACT_FLAGS_FROM_P4TC;
+	ret = tcf_action_init(net, NULL, nla, NULL, acts, init_res, &attrs_size,
+			      flags, 0, extack);
+
+	/* Check if we are trying to bind to dynamic action from different
+	 * pipeline.
+	 */
+	for (i = 0; i < TCA_ACT_MAX_PRIO && acts[i]; i++) {
+		struct tc_action *a = acts[i];
+		struct tcf_p4act *p;
+
+		if (a->ops->id <= TCA_ID_MAX)
+			continue;
+
+		p = to_p4act(a);
+		if (p->p_id != pipeid) {
+			NL_SET_ERR_MSG(extack,
+				       "Unable to bind to dynact from different pipeline");
+			ret = -EPERM;
+			goto destroy_acts;
+		}
+	}
+
+	return ret;
+
+destroy_acts:
+	p4tc_action_destroy(acts);
+	return ret;
+}
+
 struct p4tc_act *p4a_tmpl_get(struct p4tc_pipeline *pipeline,
 			      const char *act_name, const u32 a_id,
 			      struct netlink_ext_ack *extack);
@@ -181,6 +275,14 @@ static inline bool p4tc_action_put_ref(struct p4tc_act *act)
 	return refcount_dec_not_one(&act->a_ref);
 }
 
+struct tcf_p4act *
+p4a_runt_prealloc_get_next(struct p4tc_act *act);
+void p4a_runt_init_flags(struct tcf_p4act *p4act);
+void p4a_runt_parm_destroy(struct p4tc_act_param *parm);
+struct p4tc_act_param *
+p4a_runt_parm_init(struct net *net, struct p4tc_act *act,
+		   struct nlattr *nla, struct netlink_ext_ack *extack);
+
 #define to_pipeline(t) ((struct p4tc_pipeline *)t)
 #define to_hdrfield(t) ((struct p4tc_hdrfield *)t)
 #define p4tc_to_act(t) ((struct p4tc_act *)t)
include/uapi/linux/p4tc.h +13 -0
index b565b8a9e..5c2631a05 100644
@@ -115,6 +115,17 @@ enum {
 };
 
 #define P4TC_ACT_MAX (__P4TC_ACT_MAX - 1)
+
+/* Action params value attributes */
+
+enum {
+	P4TC_ACT_PARAMS_VALUE_UNSPEC,
+	P4TC_ACT_PARAMS_VALUE_RAW, /* binary */
+	__P4TC_ACT_PARAMS_VALUE_MAX
+};
+
+#define P4TC_ACT_VALUE_PARAMS_MAX (__P4TC_ACT_PARAMS_VALUE_MAX - 1)
+
 enum {
 	P4TC_ACT_PARAMS_TYPE_UNSPEC,
 	P4TC_ACT_PARAMS_TYPE_BITEND, /* u16 */
@@ -130,6 +141,8 @@ enum {
 	P4TC_ACT_PARAMS_NAME, /* string - mandatory for params create */
 	P4TC_ACT_PARAMS_ID, /* u32 */
 	P4TC_ACT_PARAMS_TYPE, /* nested type - mandatory for params create */
+	P4TC_ACT_PARAMS_VALUE, /* bytes - mandatory for runtime params create */
+	P4TC_ACT_PARAMS_MASK, /* bytes */
 	__P4TC_ACT_PARAMS_MAX
 };
 
net/sched/p4tc/p4tc_action.c +1168 -11

1,300 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#27 lore raw

PATCH v10 11/15 p4tc: add template table create, update, delete, get, flush and dump

In reply to Jamal Hadi Salim

This commit introduces code to creation and maintenance of P4 tables defined in a P4 program.

As with all other P4TC objects, tables' lifetimes conform to extended CRUD operations and are maintained via templates.
It's important to note that write operations, such as create, update and delete can only be made if the pipeline is not sealed.

Per the P4 specification, tables prefix their name with the control block (although this could be overridden by P4 annotations).

As an example, if one were to create a table named table1 in a pipeline named myprog1, on control block "mycontrol", one would use the following command:

tc p4template create table/myprog1/mycontrol/table1 tblid 1 \
keysz 32 nummasks 8 tentries 8192

Above says that we are creating a table (table1) attached to pipeline myprog1 on control block mycontrol. Table1's key size is 32 bits wide and it can have up to 8 associated masks and 8192 entries. The table id for table1 is 1. The table id is typically provided by the compiler.

Parameters such as nummasks (number of masks this table may have) and tentries (maximum number of entries this table may have); either may be omitted in which case it defaults to 8 masks and 256 entries.

P4 tables have many associated attributes that further refine how these tables operate. Some attributes per table and others per entry. Attributes include:

- Timer profiles: The timer profiles for the table entries belonging to the table. These profiles contain the supported aging values for the table. There are currently 4 different built-in profiles ID0=30s(default), ID1=60s, ID2=120s and ID3=180s.
The user can override the defaults by specifying the number of profiles per table. In that case, the kernel will generate the specified number of profiles with their aging values abiding by the following rule:
- Profile ID 0 - 30000ms
- Profile ID(n) - Profile ID(n - 1) + 30000ms
So, for example, if the user specified num_timer_profiles as 5, the profile IDs and aging values would be the following:
- Profile ID 0 - 30000ms (default profile)
- Profile ID 1 - 60000ms
- Profile ID 2 - 90000ms
- Profile ID 3 - 120000ms
- Profile ID 4 - 150000ms
The values of the different profiles could be changed at runtime. The default profile is used if the user specifies an out of range value.

- Match key type: The match type of the table key (exact, LPM, ternary, etc).
Note in P4 a key may constitute multiple (sub)key types, eg matching on srcip using a prefix and an exact match for dstip. In such a case it is up to the compiler to generalize the key used (For example in this case the overall key may endup being LPM or ternary).

- Direct Counter: Table counter instances used directly by the table, when specified in the P4 program, there will be one counter per entry.

- Direct Meter: Table meter instances used directly by the table, when specified in the P4 program, there will be one meter per entry.

- CRUDXPS Permissions both for specific entries and tables. The permissions are applicable to both the control plane and the datapath (see "Table Permissions" further below).

- Allowed Actions List. This will be a list of all possible actions that can be added to table entries that are added to the specified table.

- Action profiles. When defined in a P4 program, action profiles provide a mechanism to share action instances.

- Actions Selectors. When defined in a P4 program can be used to select the execution of one or more action instance selected at table lookup time by using a hash computation.

- Default hit action. When a default hit action is defined it is used when a matched table entry did not define an action. Depending on the P4 program the default hit action can be updated at runtime (in addition to being specified in the template).

- Default miss action. When a default miss action is defined it is used when a lookup that table fails.

- Action scope. In addition to actions being annotated as default hit or miss they can also be annotated to be either specific to a table of globally available to multiple tables within the same P4 program.

- Max entries. This is an upper bound for number of entries a specific table allows.

- Num masks. In the case of LPM or ternary matches, this defines the maximum allowed masks for that table.

- Timers. When defined in a P4 program, each entry has an associated timer. Depending on the programmed timer profile (see above), an entry gets a timeout. The timer attribute specifies the behavior of a table entry expiration.
The timer is refreshed every time there's a hit. After an idle period the P4 program can define using this attribute, whether it wants to have an event generated to user space (and have user space delete the entry), or whether it wants the kernel to delete it and send the event to announce the deletion. The default in P4TC is to both delete and generate an event.

- per entry "static" vs "dynamic" entry. By default all entries created from the control plane are "static" unless otherwise specified. All entries added from datapath are "dynamic" unless otherwise specified. "Dynamic" entries are subject to deletion when idle (subject to the rules specified in "Timers" above).

If one were to retrieve the template details of a table named table1 (before or after the pipeline is sealed) one would use the following command:

tc -j p4template get table/myprog1/mycontrol/table1 | jq .

If one were to dump all the tables from a pipeline named myprog1, one would use the following command:

tc p4template get table/myprog1

If one were to update table1 (before the pipeline is sealed) one would use the following command:

tc p4template update table/myprog1/mycontrol/table1 ....

If one were to delete table1 (before the pipeline is sealed) one would use the following command:

tc p4template del table/myprog1/mycontrol/table1

If one were to flush all the tables from a pipeline named myprog1, control block "mycontrol" one would use the following command:

tc p4template del table/myprog1/mycontrol/

___Table Permissions___

Tables can have permissions which apply to all the entries in the specified table. Permissions are defined for both what the control plane (user space) as well as the data path are allowed to do.

The permissions field is a 16bit value which will hold CRUDXPS (create, read, update, delete, execute, publish and subscribe) permissions for control and data path. Bits 13-7 will have the CRUDXPS values for control and bits 6-0 will have CRUDXPS values for data path. By default each table has the following permissions:

CRUD-PS-R--X--

Which means the control plane can perform CRUDPS operations whereas the data path can only Read and execute on the entries.
The user can override these permissions when creating the table or when updating.

For example, the following command will create a table which will not allow the datapath to create, update or delete entries but give full CRUDP permissions for the control plane.

$TC p4template create table/aP4proggie/cb/tname tblid 1 keysz 64 type lpm \
permissions 0x3D24 ...

Recall that these permissions come in the form of CRUDXPSCRUDXPS, where the first CRUDXPS block is for control and the last is for data path.

So 0x3D24 is equivalent to CR-D-P--R--X--

If we were to issue a read command on a table (tname):

$TC -j p4template get table/aP4proggie/cb/tname | jq .

The output would be the following:

[

  {
    "obj": "table",
    "pname": "aP4Proggie",
    "pipeid": 22
  },
  {
    "templates": [
      {
        "tblid": 1,
        "tname": "cb/tname",
        "keysz": 64,
        "max_entries": 256,
        "masks": 8,
        "entries": 0,
        "permissions": "CRUD-P--R--X--",
        "table_type": "lpm",
        "acts_list": []
      }
    ]

}
]

Note, the permissions concept is more powerful than classical const definition currently taken by P4 which makes everything in a table read-only.

___Initial Table Entries___

Templating can create initial table entries. For example:

tc p4template create table/myprog/cb/tname \
entry srcAddr 10.10.10.10/32 dstAddr 1.1.1.0/24 prio 17

In this command we are creating table tname with an entry which will be present before even the P4 program starts executing. This entry has as its key IPv4 srcAddr and dstAddr and prio 17.

If one was to read back the entry by issuing the following runtime command:

tc p4ctrl get myprog/table/cb/tname

They would get:
...

    entry priority 17[permissions-RUD-P--R--X--]
    entry key
        srcAddr id:1 size:32b type:ipv4 exact fieldval  10.10.10.10/32
        dstAddr id:2 size:32b type:ipv4 exact fieldval  1.1.1.0/24

We will explain the p4ctrl commands in more details on the next patches.

Before the pipeline is sealed we can only do a p4ctrl get. The write commands (create, update, delete and flush) and not allowed before sealing.

___Table Actions List___

P4 tables can be programmed to allow only a specified list of actions to be part of match entry on a table. P4 also defines default actions to be executed when no entries match; we have extended this concept to have a default hit, which is executed upon matching an entry which has no action associated with it.

We also allow flags for each of the actions in this list that specify if the action can be added only as a table entry (tableonly), or only as a default action (defaultonly). If no flags are specified, it is assumed that the action can be used in both contexts.

Both default hit and default miss are optional.

An example of specifying a default miss action is as follows:

tc p4template update table/myprog/cb/mytable \
    default_miss_action permissions 0x1124 action drop

The above will drop packets if the entry is not found in mytable. Note the above makes the default action a const. Meaning the control plane can neither replace it nor delete it.

tc p4template update table/myprog/mytable \
default_hit_action permissions 0x3004 action ok

Whereas the above allows a default hit action to accept the packet. The permission 0x3004 (binary 11000000000100) means we have only Create and Read permissions in the control plane and eXecute permissions in the data plane. This means, for example, that now we can only delete the default hit action from the control plane.

---
 include/net/p4tc.h             |  151 ++-
 include/net/p4tc_types.h       |    2 +-
 include/uapi/linux/p4tc.h      |  118 +++
 net/sched/p4tc/Makefile        |    2 +-
 net/sched/p4tc/p4tc_action.c   |    4 +-
 net/sched/p4tc/p4tc_pipeline.c |   23 +-
 net/sched/p4tc/p4tc_table.c    | 1580 ++++++++++++++++++++++++++++++++
 net/sched/p4tc/p4tc_tmpl_api.c |    2 +
 8 files changed, 1870 insertions(+), 12 deletions(-)
 create mode 100644 net/sched/p4tc/p4tc_table.c
include/net/p4tc.h +150 -1
index 7b06c63cd..51ac3a38f 100644
@@ -16,10 +16,23 @@
 #define P4TC_DEFAULT_MAX_RULES 1
 #define P4TC_PATH_MAX 3
 #define P4TC_MAX_TENTRIES 0x2000000
+#define P4TC_DEFAULT_TENTRIES 256
+#define P4TC_MAX_TMASKS 1024
+#define P4TC_DEFAULT_TMASKS 8
+#define P4TC_MAX_T_AGING_MS 864000000
+#define P4TC_DEFAULT_T_AGING_MS 30000
+#define P4TC_DEFAULT_NUM_TIMER_PROFILES 4
+#define P4TC_MAX_NUM_TIMER_PROFILES P4TC_MSGBATCH_SIZE
+
+#define P4TC_TIMER_PROFILE_ZERO_AGING_MS 30000
+#define P4TC_DEFAULT_TIMER_PROFILE_ID 0
+
+#define P4TC_MAX_PERMISSION (GENMASK(P4TC_PERM_MAX_BIT, 0))
 
 #define P4TC_KERNEL_PIPEID 0
 
 #define P4TC_PID_IDX 0
+#define P4TC_TBLID_IDX 1
 #define P4TC_AID_IDX 1
 #define P4TC_PARSEID_IDX 1
 
@@ -70,6 +83,7 @@ extern const struct p4tc_template_ops p4tc_pipeline_ops;
 struct p4tc_pipeline {
 	struct p4tc_template_common common;
 	struct idr                  p_act_idr;
+	struct idr                  p_tbl_idr;
 	struct rcu_head             rcu;
 	struct net                  *net;
 	u32                         num_created_acts;
@@ -122,6 +136,11 @@ struct p4tc_act *p4a_runt_find(struct net *net,
 void
 p4a_runt_prealloc_put(struct p4tc_act *act, struct tcf_p4act *p4_act);
 
+static inline bool p4tc_pipeline_sealed(struct p4tc_pipeline *pipeline)
+{
+	return pipeline->p_state == P4TC_STATE_READY;
+}
+
 static inline int p4tc_action_destroy(struct tc_action **acts)
 {
 	struct tc_action *acts_non_prealloc[TCA_ACT_MAX_PRIO] = {NULL};
@@ -169,6 +188,66 @@ static inline int p4tc_action_destroy(struct tc_action **acts)
 	return ret;
 }
 
+#define P4TC_CONTROL_PERMISSIONS (GENMASK(13, 7))
+#define P4TC_DATA_PERMISSIONS (GENMASK(6, 0))
+
+#define P4TC_TABLE_DEFAULT_PERMISSIONS                                   \
+	((GENMASK(P4TC_CTRL_PERM_C_BIT, P4TC_CTRL_PERM_D_BIT)) | \
+	 P4TC_CTRL_PERM_P | P4TC_CTRL_PERM_S | P4TC_DATA_PERM_R | \
+	 P4TC_DATA_PERM_X)
+
+#define P4TC_PERMISSIONS_UNINIT (1 << P4TC_PERM_MAX_BIT)
+
+struct p4tc_table_defact {
+	struct tc_action **default_acts;
+	/* Will have two 7 bits blocks containing CRUDXPS (Create, read, update,
+	 * delete, execute, publish and subscribe) permissions for control plane
+	 * and data plane. The first 5 bits are for control and the next five
+	 * are for data plane. |crudxpscrudxps| if we were to denote it as UNIX
+	 * permission flags.
+	 */
+	__u16 permissions;
+	struct rcu_head  rcu;
+};
+
+struct p4tc_table_perm {
+	__u16           permissions;
+	struct rcu_head rcu;
+};
+
+struct p4tc_table {
+	struct p4tc_template_common         common;
+	struct list_head                    tbl_acts_list;
+	struct idr                          tbl_masks_idr;
+	struct idr                          tbl_prio_idr;
+	struct xarray                       tbl_profiles_xa;
+	struct rhltable                     tbl_entries;
+	/* Mutex that protects tbl_profiles_xa */
+	struct mutex                        tbl_profiles_xa_lock;
+	struct p4tc_table_defact __rcu      *tbl_default_hitact;
+	struct p4tc_table_defact __rcu      *tbl_default_missact;
+	struct p4tc_table_perm __rcu        *tbl_permissions;
+	struct p4tc_table_entry_mask __rcu  **tbl_masks_array;
+	unsigned long __rcu                 *tbl_free_masks_bitmap;
+	/* Locks the available masks IDR which will be used when adding and
+	 * deleting table entries.
+	 */
+	spinlock_t                          tbl_masks_idr_lock;
+	u32                                 tbl_keysz;
+	u32                                 tbl_id;
+	u32                                 tbl_max_entries;
+	u32                                 tbl_max_masks;
+	u32                                 tbl_curr_num_masks;
+	atomic_t                            tbl_num_timer_profiles;
+	/* Accounts for how many entities refer to this table. Usually just the
+	 * pipeline it belongs to.
+	 */
+	refcount_t                          tbl_ctrl_ref;
+	u16                                 tbl_type;
+};
+
+extern const struct p4tc_template_ops p4tc_table_ops;
+
 struct p4tc_act_param {
 	struct list_head head;
 	struct rcu_head	rcu;
@@ -221,6 +300,20 @@ struct p4tc_act {
 	char                        fullname[ACTNAMSIZ];
 };
 
+struct p4tc_table_act {
+	struct list_head node;
+	struct tc_action_ops *ops;
+	u8     flags;
+};
+
+struct p4tc_table_timer_profile {
+	struct rcu_head rcu;
+	u64 aging_ms;
+	u32 profile_id;
+};
+
+extern struct tc_action_ops NoActionOps;
+
 extern const struct p4tc_template_ops p4tc_act_ops;
 
 static inline int p4tc_action_init(struct net *net, struct nlattr *nla,
@@ -275,6 +368,62 @@ static inline bool p4tc_action_put_ref(struct p4tc_act *act)
 	return refcount_dec_not_one(&act->a_ref);
 }
 
+struct p4tc_act_param *p4a_parm_find_byid(struct idr *params_idr,
+					  const u32 param_id);
+struct p4tc_act_param *
+p4a_parm_find_byany(struct p4tc_act *act, const char *param_name,
+		    const u32 param_id, struct netlink_ext_ack *extack);
+
+struct p4tc_table *p4tc_table_find_byany(struct p4tc_pipeline *pipeline,
+					 const char *tblname, const u32 tbl_id,
+					 struct netlink_ext_ack *extack);
+struct p4tc_table *p4tc_table_find_byid(struct p4tc_pipeline *pipeline,
+					const u32 tbl_id);
+int p4tc_table_try_set_state_ready(struct p4tc_pipeline *pipeline,
+				   struct netlink_ext_ack *extack);
+void p4tc_table_put_mask_array(struct p4tc_pipeline *pipeline);
+struct p4tc_table *p4tc_table_find_get(struct p4tc_pipeline *pipeline,
+				       const char *tblname, const u32 tbl_id,
+				       struct netlink_ext_ack *extack);
+
+static inline bool p4tc_table_put_ref(struct p4tc_table *table)
+{
+	return refcount_dec_not_one(&table->tbl_ctrl_ref);
+}
+
+struct p4tc_table_default_act_params {
+	struct p4tc_table_defact *default_hitact;
+	struct p4tc_table_defact *default_missact;
+	struct nlattr *default_hit_attr;
+	struct nlattr *default_miss_attr;
+};
+
+int
+p4tc_table_init_default_acts(struct net *net,
+			     struct p4tc_table_default_act_params *def_params,
+			     struct p4tc_table *table,
+			     struct list_head *acts_list,
+			     struct netlink_ext_ack *extack);
+
+static inline void
+p4tc_table_defacts_acts_copy(struct p4tc_table_defact *defact_copy,
+			     struct p4tc_table_defact *defact_orig)
+{
+	defact_copy->default_acts = defact_orig->default_acts;
+}
+
+void
+p4tc_table_replace_default_acts(struct p4tc_table *table,
+				struct p4tc_table_default_act_params *def_params,
+				bool lock_rtnl);
+
+struct p4tc_table_perm *
+p4tc_table_init_permissions(struct p4tc_table *table, u16 permissions,
+			    struct netlink_ext_ack *extack);
+void p4tc_table_replace_permissions(struct p4tc_table *table,
+				    struct p4tc_table_perm *tbl_perm,
+				    bool lock_rtnl);
+
 struct tcf_p4act *
 p4a_runt_prealloc_get_next(struct p4tc_act *act);
 void p4a_runt_init_flags(struct tcf_p4act *p4act);
@@ -284,7 +433,7 @@ p4a_runt_parm_init(struct net *net, struct p4tc_act *act,
 		   struct nlattr *nla, struct netlink_ext_ack *extack);
 
 #define to_pipeline(t) ((struct p4tc_pipeline *)t)
-#define to_hdrfield(t) ((struct p4tc_hdrfield *)t)
 #define p4tc_to_act(t) ((struct p4tc_act *)t)
+#define p4tc_to_table(t) ((struct p4tc_table *)t)
 
 #endif
include/net/p4tc_types.h +1 -1
index af9f51fc1..0b1d79740 100644
@@ -8,7 +8,7 @@
 
 #include <uapi/linux/p4tc.h>
 
-#define P4TC_T_MAX_BITSZ 128
+#define P4TC_T_MAX_BITSZ P4TC_MAX_KEYSZ
 
 struct p4tc_type_mask_shift {
 	void *mask;
include/uapi/linux/p4tc.h +118 -0

136 lines not on this page · raw message

net/sched/p4tc/Makefile +1 -1
index 7dbcf8915..7a9c13f86 100644
@@ -1,4 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
 
 obj-y := p4tc_types.o p4tc_tmpl_api.o p4tc_pipeline.o \
-	p4tc_action.o
+	p4tc_action.o p4tc_table.o
net/sched/p4tc/p4tc_action.c +2 -2
index c5fc52281..ec36680b6 100644
@@ -653,13 +653,13 @@ p4a_parm_find_byname(struct idr *params_idr, const char *param_name)
 	return NULL;
 }
 
-static struct p4tc_act_param *
+struct p4tc_act_param *
 p4a_parm_find_byid(struct idr *params_idr, const u32 param_id)
 {
 	return idr_find(params_idr, param_id);
 }
 
-static struct p4tc_act_param *
+struct p4tc_act_param *
 p4a_parm_find_byany(struct p4tc_act *act, const char *param_name,
 		    const u32 param_id, struct netlink_ext_ack *extack)
 {
net/sched/p4tc/p4tc_pipeline.c +16 -7
index beacdb874..82e39c3af 100644
@@ -76,6 +76,7 @@ static const struct nla_policy tc_pipeline_policy[P4TC_PIPELINE_MAX + 1] = {
 static void p4tc_pipeline_destroy(struct p4tc_pipeline *pipeline)
 {
 	idr_destroy(&pipeline->p_act_idr);
+	idr_destroy(&pipeline->p_tbl_idr);
 
 	kfree(pipeline);
 }
@@ -98,9 +99,13 @@ static void p4tc_pipeline_teardown(struct p4tc_pipeline *pipeline,
 	struct net *net = pipeline->net;
 	struct p4tc_pipeline_net *pipe_net = net_generic(net, pipeline_net_id);
 	struct net *pipeline_net = maybe_get_net(net);
-	unsigned long iter_act_id;
+	unsigned long iter_act_id, tmp;
+	struct p4tc_table *table;
 	struct p4tc_act *act;
-	unsigned long tmp;
+	unsigned long tbl_id;
+
+	idr_for_each_entry_ul(&pipeline->p_tbl_idr, table, tmp, tbl_id)
+		table->common.ops->put(pipeline, &table->common, extack);
 
 	idr_for_each_entry_ul(&pipeline->p_act_idr, act, tmp, iter_act_id)
 		act->common.ops->put(pipeline, &act->common, extack);
@@ -153,22 +158,23 @@ static int __p4tc_pipeline_put(struct p4tc_pipeline *pipeline,
 static inline int pipeline_try_set_state_ready(struct p4tc_pipeline *pipeline,
 					       struct netlink_ext_ack *extack)
 {
+	int ret;
+
 	if (pipeline->curr_tables != pipeline->num_tables) {
 		NL_SET_ERR_MSG(extack,
 			       "Must have all table defined to update state to ready");
 		return -EINVAL;
 	}
 
+	ret = p4tc_table_try_set_state_ready(pipeline, extack);
+	if (ret < 0)
+		return ret;
+
 	pipeline->p_state = P4TC_STATE_READY;
 
 	return true;
 }
 
-static inline bool p4tc_pipeline_sealed(struct p4tc_pipeline *pipeline)
-{
-	return pipeline->p_state == P4TC_STATE_READY;
-}
-
 struct p4tc_pipeline *p4tc_pipeline_find_byid(struct net *net, const u32 pipeid)
 {
 	struct p4tc_pipeline_net *pipe_net;
@@ -260,6 +266,9 @@ p4tc_pipeline_create(struct net *net, struct nlmsghdr *n,
 
 	idr_init(&pipeline->p_act_idr);
 
+	idr_init(&pipeline->p_tbl_idr);
+	pipeline->curr_tables = 0;
+
 	pipeline->num_created_acts = 0;
 
 	pipeline->p_state = P4TC_STATE_NOT_READY;
net/sched/p4tc/p4tc_table.c new file +1580 -0

1,580 lines not on this page · raw message

net/sched/p4tc/p4tc_tmpl_api.c +2 -0
index 7126ff7cd..1e79c2bde 100644
@@ -43,6 +43,7 @@ static bool obj_is_valid(u32 obj)
 	switch (obj) {
 	case P4TC_OBJ_PIPELINE:
 	case P4TC_OBJ_ACT:
+	case P4TC_OBJ_TABLE:
 		return true;
 	default:
 		return false;
@@ -52,6 +53,7 @@ static bool obj_is_valid(u32 obj)
 static const struct p4tc_template_ops *p4tc_ops[P4TC_OBJ_MAX + 1] = {
 	[P4TC_OBJ_PIPELINE] = &p4tc_pipeline_ops,
 	[P4TC_OBJ_ACT] = &p4tc_act_ops,
+	[P4TC_OBJ_TABLE] = &p4tc_table_ops,
 };
 
 int p4tc_tmpl_generic_dump(struct sk_buff *skb, struct p4tc_dump_ctx *ctx,
Signature · 1 line
2.34.1
Reviewed-by on v10 11/15
#28 lore raw

Re: [PATCH v10 net-next 11/15] p4tc: add template table create, update, delete, get, flush and dump

author maintainer
#29 lore raw

PATCH v10 12/15 p4tc: add runtime table entry create and update

In reply to Jamal Hadi Salim

Tables are conceptually similar to TCAMs and this implementation could be labelled as an "algorithmic" TCAM. Tables have a key of a specific size, maximum number of entries and masks allowed. The basic P4 key types are supported (exact, LPM, ternary, and ranges) although the kernel side is oblivious of all that and sees only bit blobs which it masks before a lookup is performed.

This commit allows users to create and update table _entries_ (templates were described in earlier patch). We'll add get, delete and dump in the next commit

Note that table entries can only be created once the pipeline template is sealed.

For example, a user issuing the following command:

tc p4ctrl create myprog/table/cb/tname \
dstAddr 10.10.10.0/24 srcAddr 192.168.0.0/16 prio 16 \
action send param port port1

indicates we are creating a table entry in table "cb/tname" (a table residing in control block "cb") on a pipeline named "myprog".

User space tc will create a key which has a value of 0x0a0a0a00c0a00000 (10.10.10.0 concatenated with 192.168.0.0) and a mask value of 0xffffff00ffff0000 (/24 concatenated with /16) that will be sent to the kernel. In addition a priority field of 16 is passed to the kernel as well as the action definition.
The priority field is needed to disambiguate in case two entries match. In that case, the kernel will choose the one with lowest priority number.

If the user wanted to, for example, update our just created entry with an action, they'd issue the following command:

tc p4ctrl update myprog/table/cb/tname srcAddr 10.10.10.0/24 \
dstAddr 192.168.0.0/16 prio 16 action send param port port5

In this case, the user needs to specify the pipeline name, the table name, the key and the priority, so that we can locate the table entry.

__Table Attributes__

Just for context, these are the table attributes (presented in the table template commit):

- Timer profiles: The timer profiles for the table entries belonging to the table. These profiles contain the supported aging values for the table. There are currently 4 different built-in profiles ID0=30s(default), ID1=60s, ID2=120s and ID3=180s.
The user can override the defaults by specifying the number of profiles per table. In that case, the kernel will generate the specified number of profiles with their aging values abiding by the following rule:
- Profile ID 0 - 30000ms
- Profile ID(n) - Profile ID(n - 1) + 30000ms
So, for example, if the user specified num_timer_profiles as 5, the profile IDs and aging values would be the following:
- Profile ID 0 - 30000ms (default profile)
- Profile ID 1 - 60000ms
- Profile ID 2 - 90000ms
- Profile ID 3 - 120000ms
- Profile ID 4 - 150000ms
The values of the different profiles could be changed at runtime. The default profile is used if the user specifies an out of range value.

- Match key type: The match type of the table key (exact, LPM, ternary, etc).
Note in P4 a key may constitute multiple (sub)key types, eg matching on srcip using a prefix and an exact match for dstip. In such a case it is up to the compiler to generalize the key used (For example in this case the overall key may endup being LPM or ternary).

- Direct Counter: Table counter instances used directly by the table, when specified in the P4 program, there will be one counter per entry.

- Direct Meter: Table meter instances used directly by the table, when specified in the P4 program, there will be one meter per entry.

- CRUDXPS Permissions both for specific entries and tables. The permissions are applicable to both the control plane and the datapath (see "Table Permissions" further below).

- Allowed Actions List. This will be a list of all possible actions that can be added to table entries that are added to the specified table.

- Action profiles. When defined in a P4 program, action profiles provide a mechanism to share action instances.

- Actions Selectors. When defined in a P4 program can be used to select the execution of one or more action instance selected at table lookup time by using a hash computation.

- Default hit action. When a default hit action is defined it is used when a matched table entry did not define an action. Depending on the P4 program the default hit action can be updated at runtime (in addition to being specified in the template).

- Default miss action. When a default miss action is defined it is used when a lookup that table fails.

- Action scope. In addition to actions being annotated as default hit or miss they can also be annotated to be either specific to a table of globally available to multiple tables within the same P4 program.

- Max entries. This is an upper bound for number of entries a specific table allows.

- Num masks. In the case of LPM or ternary matches, this defines the maximum allowed masks for that table.

- Timers. When defined in a P4 program, each entry has an associated timer. Depending on the programmed timer profile (see above), an entry gets a timeout. The timer attribute specifies the behavior of a table entry expiration.
The timer is refreshed every time there's a hit. After an idle period the P4 program can define using this attribute, whether it wants to have an event generated to user space (and have user space delete the entry), or whether it wants the kernel to delete it and send the event to announce the deletion. The default in P4TC is to both delete and generate an event.

- per entry "static" vs "dynamic" entry. By default all entries created from the control plane are "static" unless otherwise specified. All entries added from datapath are "dynamic" unless otherwise specified. "Dynamic" entries are subject to deletion when idle (subject to the rules specified in "Timers" above).

__Table Entry Permissions__

Whereas templates can define the default permissions for tables, runtime has ability to define table entry permissions as they are added.

To reiterate there are two types of permissions:
- Table permissions which are a property of the table (think directory in file systems). These are set by the template (see earlier patch on table template).
- Table entry permissions which are specific to a table entry (think a file in a directory). This patch describes those permissions.

Furthermore in both cases the permissions are split into datapath vs control path. The template definition can set either one. For example, one could allow for adding table entries by the datapath in case of PNA add-on-miss is needed.
By default tables entries have control plane RUD, meaning the control plane can Read, Update or Delete entries. By default, as well, the control plane can create new entries unless specified otherwise by the template.

Lets see an example of which creates the table "cb/tname" at template time:

    tc p4template create table/aP4proggie/cb/tname tblid 1 keysz 64 \
      permissions 0x3C24 ...

Above is setting the table tname's permission to be 0x3C24 is equivalent to CRUD----R--X-- meaning:

The control plane can Create, Read, Update, Delete The datapath can only Read and Execute table entries. If one was to dump this table with:

tc -j p4template get table/aP4proggie/cb/tname | jq .

The output would be the following:

[

  {
    "obj": "table",
    "pname": "aP4proggie",
    "pipeid": 22
  },
  {
    "templates": [
      {
        "tblid": 1,
        "tname": "cb/tname",
        "keysz": 64,
        "max_entries": 256,
        "masks": 8,
        "entries": 0,
        "permissions": "CRUD----R--X--",
        "table_type": "exact",
        "acts_list": []
      }
    ]

}
]

The expressed permissions above are probably the most practical for most use cases.

__Constant Tables And P4-programmed Defined Entries__

If one wanted to restrict the table to be an equivalent to a "const" then the permissions would be set to be: -R------R--X-- by the template.

In such a case, typically the P4 program will have some entries defined (see the famous P4 calculate example). The "initial entries" specified in the P4 program will have to be added by the template (as generated by the compiler), as such:

tc p4template create table/aP4proggie/cb/tname \
entry srcAddr 10.10.10.10/24 dstAddr 1.1.1.0/24 prio 17

This table cannot be updated at runtime. Any attempt to add an entry of a table which is read-only at runtime will get a "permission denied" response back from the kernel.

Note: If one was to create an equivalent for PNA add-on-miss feature for this table, then the template would issue table permissions as:
-R-----CR--X-- PNA doesn't specify whether the datapath can also delete or update entries, but if it did then more appropriate permissions will be:
-R-----CRUDX--

__Mix And Match of RW vs Constant Entries__
Lets look at other scenarios; lets say the table has CRUD----R--X-- permissions as defined by the template...
At runtime the user could add entries which are "const" - by specifying the entry's permission as -R------R--X-- example:

tc p4ctrl create aP4proggie/table/cb/tname srcAddr 10.10.10.10/24 \
dstAddr 1.1.1.0/24 prio 17 permissions 0x1024 action drop

or not specify permissions at all as such:

tc p4ctrl create aP4proggie/table/cb/tname srcAddr 10.10.10.10/24 \
dstAddr 1.1.1.0/24 prio 17 action drop

in which case the table's permissions defined at template time(CRUD----R--X--) are assumed; meaning the table entry can be deleted or updated by the control plane.

__Entries permissions Allowed On A Table Entry Creation At Runtime__

When entries are created without specifying permissions they inherit the table permissions set by the template.
When an entry is added with expressed permissions it has at most to have what the template table definition expressed but could ask for less permission. For example, assuming a table with templated specified permissions of CR-D----R--X--:
An entry created at runtime with permission of -R------R--X-- is allowed but an entry with -RUD----R--X-- will be rejected.

---
 include/net/p4tc.h                |  121 +-
 include/uapi/linux/p4tc.h         |   73 +-
 include/uapi/linux/rtnetlink.h    |    9 +
 net/sched/p4tc/Makefile           |    3 +-
 net/sched/p4tc/p4tc_runtime_api.c |   82 ++
 net/sched/p4tc/p4tc_table.c       |  100 +-
 net/sched/p4tc/p4tc_tbl_entry.c   | 1805 +++++++++++++++++++++++++++++
 net/sched/p4tc/p4tc_tmpl_api.c    |    4 +-
 security/selinux/nlmsgtab.c       |    6 +-
 9 files changed, 2186 insertions(+), 17 deletions(-)
 create mode 100644 net/sched/p4tc/p4tc_runtime_api.c
 create mode 100644 net/sched/p4tc/p4tc_tbl_entry.c
include/net/p4tc.h +119 -2
index 51ac3a38f..e0b1578c6 100644
@@ -124,6 +124,11 @@ static inline bool p4tc_pipeline_get(struct p4tc_pipeline *pipeline)
 	return refcount_inc_not_zero(&pipeline->p_ctrl_ref);
 }
 
+static inline void p4tc_pipeline_put_ref(struct p4tc_pipeline *pipeline)
+{
+	refcount_dec(&pipeline->p_ctrl_ref);
+}
+
 void p4tc_pipeline_put(struct p4tc_pipeline *pipeline);
 struct p4tc_pipeline *
 p4tc_pipeline_find_byany_unsealed(struct net *net, const char *p_name,
@@ -198,6 +203,8 @@ static inline int p4tc_action_destroy(struct tc_action **acts)
 
 #define P4TC_PERMISSIONS_UNINIT (1 << P4TC_PERM_MAX_BIT)
 
+#define P4TC_MAX_PARAM_DATA_SIZE 124
+
 struct p4tc_table_defact {
 	struct tc_action **default_acts;
 	/* Will have two 7 bits blocks containing CRUDXPS (Create, read, update,
@@ -219,11 +226,12 @@ struct p4tc_table {
 	struct p4tc_template_common         common;
 	struct list_head                    tbl_acts_list;
 	struct idr                          tbl_masks_idr;
-	struct idr                          tbl_prio_idr;
+	struct ida                          tbl_prio_idr;
 	struct xarray                       tbl_profiles_xa;
 	struct rhltable                     tbl_entries;
 	/* Mutex that protects tbl_profiles_xa */
 	struct mutex                        tbl_profiles_xa_lock;
+	struct p4tc_table_entry             *tbl_entry;
 	struct p4tc_table_defact __rcu      *tbl_default_hitact;
 	struct p4tc_table_defact __rcu      *tbl_default_missact;
 	struct p4tc_table_perm __rcu        *tbl_permissions;
@@ -239,11 +247,14 @@ struct p4tc_table {
 	u32                                 tbl_max_masks;
 	u32                                 tbl_curr_num_masks;
 	atomic_t                            tbl_num_timer_profiles;
+	/* Accounts for how many entries this table has */
+	atomic_t                            tbl_nelems;
 	/* Accounts for how many entities refer to this table. Usually just the
 	 * pipeline it belongs to.
 	 */
 	refcount_t                          tbl_ctrl_ref;
 	u16                                 tbl_type;
+	u16                                 PAD0;
 };
 
 extern const struct p4tc_template_ops p4tc_table_ops;
@@ -316,6 +327,84 @@ extern struct tc_action_ops NoActionOps;
 
 extern const struct p4tc_template_ops p4tc_act_ops;
 
+extern const struct rhashtable_params entry_hlt_params;
+
+struct p4tc_table_entry;
+struct p4tc_table_entry_work {
+	struct work_struct   work;
+	struct p4tc_pipeline *pipeline;
+	struct p4tc_table_entry *entry;
+	struct p4tc_table *table;
+	u16 who_deleted;
+	bool send_event;
+};
+
+struct p4tc_table_entry_key {
+	u32 keysz;
+	/* Key start */
+	u32 maskid;
+	unsigned char fa_key[] __aligned(8);
+};
+
+struct p4tc_table_entry_value {
+	u32                              prio;
+	int                              num_acts;
+	struct tc_action                 **acts;
+	/* Accounts for how many entities are referencing, eg: Data path,
+	 * one or more control path and timer.
+	 */
+	refcount_t                       entries_ref;
+	u32                              permissions;
+	struct p4tc_table_entry_tm __rcu *tm;
+	struct p4tc_table_entry_work     *entry_work;
+	bool                             tmpl_created;
+};
+
+struct p4tc_table_entry_mask {
+	struct rcu_head	 rcu;
+	u32              sz;
+	u32              mask_index;
+	/* Accounts for how many entries are using this mask */
+	refcount_t       mask_ref;
+	u32              mask_id;
+	unsigned char fa_value[] __aligned(8);
+};
+
+struct p4tc_table_entry {
+	struct rcu_head rcu;
+	struct rhlist_head ht_node;
+	struct p4tc_table_entry_key key;
+	/* fallthrough: key data + value */
+};
+
+#define P4TC_KEYSZ_BYTES(bits) (round_up(BITS_TO_BYTES(bits), 8))
+
+#define P4TC_ENTRY_KEY_OFFSET (offsetof(struct p4tc_table_entry_key, fa_key))
+
+#define P4TC_ENTRY_VALUE_OFFSET(entry) \
+	(offsetof(struct p4tc_table_entry, key) + P4TC_ENTRY_KEY_OFFSET \
+	 + P4TC_KEYSZ_BYTES((entry)->key.keysz))
+
+static inline void *p4tc_table_entry_value(struct p4tc_table_entry *entry)
+{
+	return entry->key.fa_key + P4TC_KEYSZ_BYTES(entry->key.keysz);
+}
+
+static inline struct p4tc_table_entry_work *
+p4tc_table_entry_work(struct p4tc_table_entry *entry)
+{
+	struct p4tc_table_entry_value *value = p4tc_table_entry_value(entry);
+
+	return value->entry_work;
+}
+
+extern const struct nla_policy p4tc_root_policy[P4TC_ROOT_MAX + 1];
+extern const struct nla_policy p4tc_policy[P4TC_MAX + 1];
+
+struct p4tc_table_entry *
+p4tc_table_entry_lookup_direct(struct p4tc_table *table,
+			       struct p4tc_table_entry_key *key);
+
 static inline int p4tc_action_init(struct net *net, struct nlattr *nla,
 				   struct tc_action *acts[], u32 pipeid,
 				   u32 flags, struct netlink_ext_ack *extack)
@@ -405,6 +494,14 @@ p4tc_table_init_default_acts(struct net *net,
 			     struct list_head *acts_list,
 			     struct netlink_ext_ack *extack);
 
+static inline void p4tc_table_defact_destroy(struct p4tc_table_defact *defact)
+{
+	if (defact) {
+		p4tc_action_destroy(defact->default_acts);
+		kfree(defact);
+	}
+}
+
 static inline void
 p4tc_table_defacts_acts_copy(struct p4tc_table_defact *defact_copy,
 			     struct p4tc_table_defact *defact_orig)
@@ -419,10 +516,25 @@ p4tc_table_replace_default_acts(struct p4tc_table *table,
 
 struct p4tc_table_perm *
 p4tc_table_init_permissions(struct p4tc_table *table, u16 permissions,
-			    struct netlink_ext_ack *extack);
+			   struct netlink_ext_ack *extack);
 void p4tc_table_replace_permissions(struct p4tc_table *table,
 				    struct p4tc_table_perm *tbl_perm,
 				    bool lock_rtnl);
+void p4tc_table_entry_destroy_hash(void *ptr, void *arg);
+
+struct p4tc_table_entry *
+p4tc_tmpl_table_entry_cu(struct net *net, struct nlattr *arg,
+			 struct p4tc_pipeline *pipeline,
+			 struct p4tc_table *table,
+			 struct netlink_ext_ack *extack);
+int p4tc_tbl_entry_root(struct net *net, struct sk_buff *skb,
+			struct nlmsghdr *n, int cmd,
+			struct netlink_ext_ack *extack);
+int p4tc_tbl_entry_dumpit(struct net *net, struct sk_buff *skb,
+			  struct netlink_callback *cb,
+			  struct nlattr *arg, char *p_name);
+int p4tc_tbl_entry_fill(struct sk_buff *skb, struct p4tc_table *table,
+			struct p4tc_table_entry *entry, u32 tbl_id);
 
 struct tcf_p4act *
 p4a_runt_prealloc_get_next(struct p4tc_act *act);
@@ -432,6 +544,11 @@ struct p4tc_act_param *
 p4a_runt_parm_init(struct net *net, struct p4tc_act *act,
 		   struct nlattr *nla, struct netlink_ext_ack *extack);
 
+static inline bool p4tc_runtime_msg_is_update(struct nlmsghdr *n)
+{
+	return n->nlmsg_type == RTM_P4TC_UPDATE;
+}
+
 #define to_pipeline(t) ((struct p4tc_pipeline *)t)
 #define p4tc_to_act(t) ((struct p4tc_act *)t)
 #define p4tc_to_table(t) ((struct p4tc_table *)t)
include/uapi/linux/p4tc.h +72 -1

103 lines not on this page · raw message

include/uapi/linux/rtnetlink.h +9 -0
index 4f9ebe3e7..76645560b 100644
@@ -203,6 +203,15 @@ enum {
 	RTM_UPDATEP4TEMPLATE,
 #define RTM_UPDATEP4TEMPLATE	RTM_UPDATEP4TEMPLATE
 
+	RTM_P4TC_CREATE = 128,
+#define RTM_P4TC_CREATE	RTM_P4TC_CREATE
+	RTM_P4TC_DEL,
+#define RTM_P4TC_DEL		RTM_P4TC_DEL
+	RTM_P4TC_GET,
+#define RTM_P4TC_GET		RTM_P4TC_GET
+	RTM_P4TC_UPDATE,
+#define RTM_P4TC_UPDATE	RTM_P4TC_UPDATE
+
 	__RTM_MAX,
 #define RTM_MAX		(((__RTM_MAX + 3) & ~3) - 1)
 };
net/sched/p4tc/Makefile +2 -1
index 7a9c13f86..921909ac4 100644
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
 
 obj-y := p4tc_types.o p4tc_tmpl_api.o p4tc_pipeline.o \
-	p4tc_action.o p4tc_table.o
+	p4tc_action.o p4tc_table.o p4tc_tbl_entry.o \
+	p4tc_runtime_api.o
net/sched/p4tc/p4tc_runtime_api.c new file +82 -0

82 lines not on this page · raw message

net/sched/p4tc/p4tc_table.c +90 -10

174 lines not on this page · raw message

net/sched/p4tc/p4tc_tbl_entry.c new file +1805 -0

1,805 lines not on this page · raw message

net/sched/p4tc/p4tc_tmpl_api.c +2 -2
index 1e79c2bde..17232060a 100644
@@ -27,12 +27,12 @@
 #include <net/netlink.h>
 #include <net/flow_offload.h>
 
-static const struct nla_policy p4tc_root_policy[P4TC_ROOT_MAX + 1] = {
+const struct nla_policy p4tc_root_policy[P4TC_ROOT_MAX + 1] = {
 	[P4TC_ROOT] = { .type = NLA_NESTED },
 	[P4TC_ROOT_PNAME] = { .type = NLA_STRING, .len = P4TC_PIPELINE_NAMSIZ },
 };
 
-static const struct nla_policy p4tc_policy[P4TC_MAX + 1] = {
+const struct nla_policy p4tc_policy[P4TC_MAX + 1] = {
 	[P4TC_PATH] = { .type = NLA_BINARY,
 			.len = P4TC_PATH_MAX * sizeof(u32) },
 	[P4TC_PARAMS] = { .type = NLA_NESTED },
security/selinux/nlmsgtab.c +5 -1
index e50a1c1ff..da7902404 100644
@@ -98,6 +98,10 @@ static const struct nlmsg_perm nlmsg_route_perms[] = {
 	{ RTM_DELP4TEMPLATE,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
 	{ RTM_GETP4TEMPLATE,	NETLINK_ROUTE_SOCKET__NLMSG_READ },
 	{ RTM_UPDATEP4TEMPLATE,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
+	{ RTM_P4TC_CREATE,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
+	{ RTM_P4TC_DEL,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
+	{ RTM_P4TC_GET,	NETLINK_ROUTE_SOCKET__NLMSG_READ },
+	{ RTM_P4TC_UPDATE,	NETLINK_ROUTE_SOCKET__NLMSG_WRITE },
 };
 
 static const struct nlmsg_perm nlmsg_tcpdiag_perms[] = {
@@ -181,7 +185,7 @@ int selinux_nlmsg_lookup(u16 sclass, u16 nlmsg_type, u32 *perm)
 		 * structures at the top of this file with the new mappings
 		 * before updating the BUILD_BUG_ON() macro!
 		 */
-		BUILD_BUG_ON(RTM_MAX != (RTM_CREATEP4TEMPLATE + 3));
+		BUILD_BUG_ON(RTM_MAX != (RTM_P4TC_CREATE + 3));
 		err = nlmsg_perm(nlmsg_type, perm, nlmsg_route_perms,
 				 sizeof(nlmsg_route_perms));
 		break;
Signature · 1 line
2.34.1
Reviewed-by on v10 12/15
#30 lore raw

Re: [PATCH v10 net-next 12/15] p4tc: add runtime table entry create and update

author maintainer
#31 lore raw

PATCH v10 13/15 p4tc: add runtime table entry get, delete, flush and dump

In reply to Jamal Hadi Salim

This commit allows users to get, delete, flush and dump table _entries_ (templates were described in earlier patch and so were create and update).

If the user wants to, for example, read a table entry from a table (nh_table) which has as key nh_index (bit32), belongs to pipeline (routing) and has as its key an nh_index of 1, they'd issue the following command:

tc p4ctrl get routing/table/Main/nh_table nh_index 1

Which will give us the following output:

$TC p4ctrl get routing/table/Main/nh_table
pipeline: routing(id 1)
table: Main/nh_table(id 1)entry priority 64000[permissions -RUD-PS-R--X--]
entry key
nh_index id:1 size:32b type:bit32 exact fieldval 1
entry actions:
action order 1: routing/Main/set_nh index 2 ref 1 bind 1
params:
dmac type macaddr value: 13:37:13:37:13:37 id 1
port type dev value: port1 id 2

created by: tc (id 2)
dynamic false
created 20338 sec used 39 sec

Note that, as with create and update, we need to specify the pipeline name, the table name, the key and the priority, so that we can locate the table entry. Also, in this case, the entry had an action which has a parameter dmac (a mac address) and port1 (a net device).

If the user wanted to delete the same table entry, they'd issue the following command:

tc p4ctrl del routing/table/Main/nh_table nh_index 1

Note that, again, we need to specify the pipeline name, the table name, the key and the priority, so that we can
locate the table entry.

We can also flush all the table entries from a specific table. To flush the table entries of table tname ane pipeline ptables, the user would issue the following command:

tc p4ctrl del routing/table/Main/nh_table

We can also dump all the table entries from a specific table. To dump the table entries of table tname and pipeline myprog, the user would issue the following command:

tc p4ctrl get routing/table/Main/nh_table

Which will give the following output (in case we had one entry with nh_index key value 1):

pipeline: routing(id 1)
table: Main/nh_table(id 1)entry priority 64000[permissions -RUD-PS-R--X--]
entry key
nh_index id:1 size:32b type:bit32 exact fieldval 1
entry actions:
action order 1: routing/Main/set_nh index 2 ref 1 bind 1
params:
dmac type macaddr value: 13:37:13:37:13:37 id 1
port type dev value: port1 id 2

created by: tc (id 2)
dynamic false
created 20338 sec used 39 sec

---
 include/net/p4tc.h                |  30 +-
 include/uapi/linux/p4tc.h         |  90 ++-
 net/sched/p4tc/Makefile           |   2 +-
 net/sched/p4tc/p4tc_filter.c      | 872 ++++++++++++++++++++++++++++++
 net/sched/p4tc/p4tc_runtime_api.c |  63 +++
 net/sched/p4tc/p4tc_table.c       | 113 +++-
 net/sched/p4tc/p4tc_tbl_entry.c   | 856 ++++++++++++++++++++++++++++-
 7 files changed, 2015 insertions(+), 11 deletions(-)
 create mode 100644 net/sched/p4tc/p4tc_filter.c
include/net/p4tc.h +29 -1
index e0b1578c6..21cc818ce 100644
@@ -36,9 +36,15 @@
 #define P4TC_AID_IDX 1
 #define P4TC_PARSEID_IDX 1
 
+struct p4tc_filter {
+	struct p4tc_filter_oper *operation;
+	int cmd; /* CRUD command */
+};
+
 struct p4tc_dump_ctx {
 	u32 ids[P4TC_PATH_MAX];
 	struct rhashtable_iter *iter;
+	struct p4tc_filter *entry_filter;
 };
 
 struct p4tc_template_common;
@@ -357,6 +363,9 @@ struct p4tc_table_entry_value {
 	u32                              permissions;
 	struct p4tc_table_entry_tm __rcu *tm;
 	struct p4tc_table_entry_work     *entry_work;
+	u64                              aging_ms;
+	struct hrtimer                   entry_timer;
+	bool                             is_dyn;
 	bool                             tmpl_created;
 };
 
@@ -520,6 +529,15 @@ p4tc_table_init_permissions(struct p4tc_table *table, u16 permissions,
 void p4tc_table_replace_permissions(struct p4tc_table *table,
 				    struct p4tc_table_perm *tbl_perm,
 				    bool lock_rtnl);
+int p4tc_table_timer_profile_update(struct p4tc_table *table,
+				    struct nlattr *nla,
+				    struct netlink_ext_ack *extack);
+struct p4tc_table_timer_profile *
+p4tc_table_timer_profile_find_byaging(struct p4tc_table *table,
+				      u64 aging_ms);
+struct p4tc_table_timer_profile *
+p4tc_table_timer_profile_find(struct p4tc_table *table, u32 profile_id);
+
 void p4tc_table_entry_destroy_hash(void *ptr, void *arg);
 
 struct p4tc_table_entry *
@@ -534,7 +552,17 @@ int p4tc_tbl_entry_dumpit(struct net *net, struct sk_buff *skb,
 			  struct netlink_callback *cb,
 			  struct nlattr *arg, char *p_name);
 int p4tc_tbl_entry_fill(struct sk_buff *skb, struct p4tc_table *table,
-			struct p4tc_table_entry *entry, u32 tbl_id);
+			struct p4tc_table_entry *entry, u32 tbl_id,
+			u16 who_deleted);
+void p4tc_tbl_entry_mask_key(u8 *masked_key, u8 *key, const u8 *mask,
+			     u32 masksz);
+
+struct p4tc_filter *
+p4tc_filter_build(struct p4tc_pipeline *pipeline, struct p4tc_table *table,
+		  struct nlattr *nla, struct netlink_ext_ack *extack);
+bool p4tc_filter_exec(struct p4tc_filter *filter,
+		      struct p4tc_table_entry *entry);
+void p4tc_filter_destroy(struct p4tc_filter *filter);
 
 struct tcf_p4act *
 p4a_runt_prealloc_get_next(struct p4tc_act *act);
include/uapi/linux/p4tc.h +89 -1

119 lines not on this page · raw message

net/sched/p4tc/Makefile +1 -1
index 921909ac4..56a8adc74 100644
@@ -2,4 +2,4 @@
 
 obj-y := p4tc_types.o p4tc_tmpl_api.o p4tc_pipeline.o \
 	p4tc_action.o p4tc_table.o p4tc_tbl_entry.o \
-	p4tc_runtime_api.o
+	p4tc_filter.o p4tc_runtime_api.o
net/sched/p4tc/p4tc_filter.c new file +872 -0

872 lines not on this page · raw message

net/sched/p4tc/p4tc_runtime_api.c +63 -0

81 lines not on this page · raw message

net/sched/p4tc/p4tc_table.c +112 -1

131 lines not on this page · raw message

net/sched/p4tc/p4tc_tbl_entry.c +849 -7

1,011 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v10 13/15
#32 lore raw

Re: [PATCH v10 net-next 13/15] p4tc: add runtime table entry get, delete, flush and dump

author maintainer
#33 lore raw

PATCH v10 14/15 p4tc: add set of P4TC table kfuncs

In reply to Jamal Hadi Salim

We add an initial set of kfuncs to allow interactions from eBPF programs to the P4TC domain.

- bpf_p4tc_tbl_read: Used to lookup a table entry from a BPF program installed in TC. To find the table entry we take in an skb, the pipeline ID, the table ID, a key and a key size.
We use the skb to get the network namespace structure where all the pipelines are stored. After that we use the pipeline ID and the table ID, to find the table. We then use the key to search for the entry. We return an entry on success and NULL on failure.

- xdp_p4tc_tbl_read: Used to lookup a table entry from a BPF program installed in XDP. To find the table entry we take in an xdp_md, the pipeline ID, the table ID, a key and a key size.
We use struct xdp_md to get the network namespace structure where all the pipelines are stored. After that we use the pipeline ID and the table ID, to find the table. We then use the key to search for the entry. We return an entry on success and NULL on failure.

- bpf_p4tc_entry_create: Used to create a table entry from a BPF program installed in TC. To create the table entry we take an skb, the pipeline ID, the table ID, a key and its size, and an action which will be associated with the new entry.
We return 0 on success and a negative errno on failure

- xdp_p4tc_entry_create: Used to create a table entry from a BPF program installed in XDP. To create the table entry we take an xdp_md, the pipeline ID, the table ID, a key and its size, and an action which will be associated with the new entry.
We return 0 on success and a negative errno on failure

- bpf_p4tc_entry_create_on_miss: conforms to PNA "add on miss". First does a lookup using the passed key and upon a miss will add the entry to the table.
We return 0 on success and a negative errno on failure

- xdp_p4tc_entry_create_on_miss: conforms to PNA "add on miss". First does a lookup using the passed key and upon a miss will add the entry to the table.
We return 0 on success and a negative errno on failure

- bpf_p4tc_entry_update: Used to update a table entry from a BPF program installed in TC. To update the table entry we take an skb, the pipeline ID, the table ID, a key and its size, and an action which will be associated with the new entry.
We return 0 on success and a negative errno on failure

- xdp_p4tc_entry_update: Used to update a table entry from a BPF program installed in XDP. To update the table entry we take an xdp_md, the pipeline ID, the table ID, a key and its size, and an action which will be associated with the new entry.
We return 0 on success and a negative errno on failure

- bpf_p4tc_entry_delete: Used to delete a table entry from a BPF program installed in TC. To delete the table entry we take an skb, the pipeline ID, the table ID, a key and a key size.
We return 0 on success and a negative errno on failure

- xdp_p4tc_entry_delete: Used to delete a table entry from a BPF program installed in XDP. To delete the table entry we take an xdp_md, the pipeline ID, the table ID, a key and a key size.
We return 0 on success and a negative errno on failure

---
 include/linux/bitops.h          |   1 +
 include/net/p4tc.h              |  60 ++++-
 include/net/tc_act/p4tc.h       |  24 ++
 include/uapi/linux/p4tc.h       |   2 +
 net/sched/p4tc/Makefile         |   1 +
 net/sched/p4tc/p4tc_action.c    |  70 +++++-
 net/sched/p4tc/p4tc_bpf.c       | 338 ++++++++++++++++++++++++++
 net/sched/p4tc/p4tc_pipeline.c  |  47 +++-
 net/sched/p4tc/p4tc_table.c     |   8 +
 net/sched/p4tc/p4tc_tbl_entry.c | 407 +++++++++++++++++++++++++++++++-
 net/sched/p4tc/p4tc_tmpl_api.c  |   4 +
 11 files changed, 953 insertions(+), 9 deletions(-)
 create mode 100644 net/sched/p4tc/p4tc_bpf.c
include/linux/bitops.h +1 -0
index 2ba557e06..290c2399a 100644
@@ -19,6 +19,7 @@
 #define BITS_TO_LONGS(nr)	__KERNEL_DIV_ROUND_UP(nr, BITS_PER_TYPE(long))
 #define BITS_TO_U64(nr)		__KERNEL_DIV_ROUND_UP(nr, BITS_PER_TYPE(u64))
 #define BITS_TO_U32(nr)		__KERNEL_DIV_ROUND_UP(nr, BITS_PER_TYPE(u32))
+#define BITS_TO_U16(nr)		__KERNEL_DIV_ROUND_UP(nr, BITS_PER_TYPE(u16))
 #define BITS_TO_BYTES(nr)	__KERNEL_DIV_ROUND_UP(nr, BITS_PER_TYPE(char))
 
 extern unsigned int __sw_hweight8(unsigned int w);
include/net/p4tc.h +59 -1

91 lines not on this page · raw message

include/net/tc_act/p4tc.h +24 -0
index 6447fe5ce..ca925d112 100644
@@ -14,10 +14,23 @@ struct tcf_p4act_params {
 	u32 tot_params_sz;
 };
 
+#define P4TC_MAX_PARAM_DATA_SIZE 124
+
+struct p4tc_table_entry_act_bpf {
+	u32 act_id;
+	u8 params[P4TC_MAX_PARAM_DATA_SIZE];
+} __packed;
+
+struct p4tc_table_entry_act_bpf_kern {
+	struct rcu_head rcu;
+	struct p4tc_table_entry_act_bpf act_bpf;
+};
+
 struct tcf_p4act {
 	struct tc_action common;
 	/* Params IDR reference passed during runtime */
 	struct tcf_p4act_params __rcu *params;
+	struct p4tc_table_entry_act_bpf_kern __rcu *act_bpf;
 	u32 p_id;
 	u32 act_id;
 	struct list_head node;
@@ -25,4 +38,15 @@ struct tcf_p4act {
 
 #define to_p4act(a) ((struct tcf_p4act *)a)
 
+static inline struct p4tc_table_entry_act_bpf *
+p4tc_table_entry_act_bpf(struct tc_action *action)
+{
+	struct p4tc_table_entry_act_bpf_kern *act_bpf;
+	struct tcf_p4act *p4act = to_p4act(action);
+
+	act_bpf = rcu_dereference(p4act->act_bpf);
+
+	return &act_bpf->act_bpf;
+}
+
 #endif /* __NET_TC_ACT_P4_H */
include/uapi/linux/p4tc.h +2 -0
index 4df79394d..c92aa106c 100644
@@ -19,6 +19,8 @@ struct p4tcmsg {
 #define P4TC_MINTABLES_COUNT 0
 #define P4TC_MSGBATCH_SIZE 16
 
+#define P4TC_ACT_MAX_NUM_PARAMS P4TC_MSGBATCH_SIZE
+
 #define P4TC_MAX_KEYSZ 512
 #define P4TC_DEFAULT_NUM_PREALLOC 16
 
net/sched/p4tc/Makefile +1 -0
index 56a8adc74..73ccb53c4 100644
@@ -3,3 +3,4 @@
 obj-y := p4tc_types.o p4tc_tmpl_api.o p4tc_pipeline.o \
 	p4tc_action.o p4tc_table.o p4tc_tbl_entry.o \
 	p4tc_filter.o p4tc_runtime_api.o
+obj-$(CONFIG_DEBUG_INFO_BTF) += p4tc_bpf.o
net/sched/p4tc/p4tc_action.c +69 -1

132 lines not on this page · raw message

net/sched/p4tc/p4tc_bpf.c new file +338 -0

338 lines not on this page · raw message

net/sched/p4tc/p4tc_pipeline.c +45 -2

65 lines not on this page · raw message

net/sched/p4tc/p4tc_table.c +8 -0
index bcf29b462..b04614fd8 100644
@@ -636,6 +636,7 @@ static int _p4tc_table_put(struct net *net, struct nlattr **tb,
 
 	rhltable_free_and_destroy(&table->tbl_entries,
 				  p4tc_table_entry_destroy_hash, table);
+	p4tc_tbl_cache_remove(net, table);
 
 	idr_destroy(&table->tbl_masks_idr);
 	ida_destroy(&table->tbl_prio_idr);
@@ -1402,6 +1403,10 @@ static struct p4tc_table *p4tc_table_create(struct net *net, struct nlattr **tb,
 		goto profiles_destroy;
 	}
 
+	ret = p4tc_tbl_cache_insert(net, pipeline->common.p_id, table);
+	if (ret < 0)
+		goto entries_hashtable_destroy;
+
 	pipeline->curr_tables += 1;
 
 	table->common.ops = (struct p4tc_template_ops *)&p4tc_table_ops;
@@ -1409,6 +1414,9 @@ static struct p4tc_table *p4tc_table_create(struct net *net, struct nlattr **tb,
 
 	return table;
 
+entries_hashtable_destroy:
+	rhltable_destroy(&table->tbl_entries);
+
 profiles_destroy:
 	p4tc_table_timer_profiles_destroy(table);
 
net/sched/p4tc/p4tc_tbl_entry.c +402 -5

449 lines not on this page · raw message

net/sched/p4tc/p4tc_tmpl_api.c +4 -0
index 17232060a..6529ff280 100644
@@ -599,6 +599,10 @@ static int __init p4tc_template_init(void)
 			op->init();
 	}
 
+#if IS_ENABLED(CONFIG_DEBUG_INFO_BTF)
+	register_p4tc_tbl_bpf();
+#endif
+
 	return 0;
 }
 
Signature · 1 line
2.34.1
author maintainer
#35 lore raw

PATCH v10 15/15 p4tc: add P4 classifier

In reply to Jamal Hadi Salim

Introduce P4 tc classifier. The main task of this classifier is to manage the lifetime of pipeline instances across one or more netdev ports. Note a pipeline may be instantiated multiple times across one or more tc chains and different priorities.

Note that part or whole of the P4 pipeline could reside in tc, XDP or even hardware depending on how the P4 program was compiled.
To use the P4 classifier you must specify a pipeline name that will be associated to the filter instance, a s/w parser (eBPF) and datapath P4 control block program (eBPF) program. Although this patchset does not deal with offloads, it is also possible to load the h/w part using this filter. We will illustrate a few examples further below to clarify. Please treat the illustrated split as an example - there are probably more pragmatic approaches to splitting the pipeline; however, regardless of where the different pieces of the pipeline are placed (tc, XDP, HW) and what each layer will implement (what part of the pipeline) - these examples are merely showing what is possible.

The pipeline is assumed to have already been created via a template.

For example, if we were to add a filter to ingress of a group of netdevs (tc block 22) and associate it to P4 pipeline simple_l3 we could issue the following command:

tc filter add block 22 parent ffff: protocol all prio 6 p4 pname simple_l3 \
action bpf obj $PARSER.o ... \
action bpf obj $PROGNAME.o section prog/tc-ingress

The above uses the classical tc action mechanism in which the first action runs the P4 parser and if that goes well then the P4 control block is executed. Note, although not shown above, one could also append the command line with other traditional tc actions.

In these patches, we also support two types of loadings of the pipeline programs and differentiate between what gets loaded at say tc vs xdp by using syntax which specifies location as either "prog type tc obj" or "prog type xdp obj". There is an ongoing discussion in the P4TC community biweekly meetings which is likely going to have us add another location definition "prog type hw" which will specify the hardware object file name and other related attributes.

An example using tc:

tc filter add block 22 parent ffff: protocol all prio 6 p4 pname simple_l3 \
prog type tc obj $PARSER.o ... \
action bpf obj $PROGNAME.o section prog/tc-ingress

For XDP, to illustrate an example:

tc filter add dev $P0 ingress protocol all prio 1 p4 pname simple_l3 \

    prog type xdp obj $PARSER.o section parser/xdp \
    pinned_link /sys/fs/bpf/mylink \
    action bpf obj $PROGNAME.o section prog/tc-ingress

In this case, the parser will be executed in the XDP layer and the rest of P4 control block as a tc action.

For illustration sake, the hw one looks as follows (please note there's still a lot of discussions going on in the meetings - the example is here merely to illustrate the tc filter functionality):

tc filter add block 22 ingress protocol all prio 1 p4 pname simple_l3 \
prog type hw filename "mypnameprog.o" ... \
prog type xdp obj $PARSER.o section parser/xdp pinned_link /sys/fs/bpf/mylink \
action bpf obj $PROGNAME.o section prog/tc-ingress

The theory of operations is as follows:

================================1. PARSING================================

The packet first encounters the parser.
The parser is implemented in ebpf residing either at the TC or XDP level. The parsed header values are stored in a shared eBPF map. When the parser runs at XDP level, we load it into XDP using tc filter command and pin it to a file.

=============================2. ACTIONS=============================

In the above example, the P4 program (minus the parser) is encoded in an action($PROGNAME.o). It should be noted that classical tc actions continue to work:
IOW, someone could decide to add a mirred action to mirror all packets after or before the ebpf action.

tc filter add dev $P0 parent ffff: protocol all prio 6 p4 pname simple_l3 \

    prog type tc obj $PARSER.o section parser/tc-ingress \
    action bpf obj $PROGNAME.o section prog/tc-ingress \
    action mirred egress mirror index 1 dev $P1 \
    action bpf obj $ANOTHERPROG.o section mysect/section-1

It should also be noted that it is feasible to split some of the ingress datapath into XDP first and more into TC later (as was shown above for example where the parser runs at XDP level). YMMV. Regardless of choice of which scheme to use, none of these will affect UAPI. It will all depend on whether you generate code to load on XDP vs tc, etc.

---
 include/uapi/linux/pkt_cls.h |  18 ++
 net/sched/Kconfig            |  12 +
 net/sched/Makefile           |   1 +
 net/sched/cls_p4.c           | 450 +++++++++++++++++++++++++++++++++++
 net/sched/p4tc/Makefile      |   4 +-
 net/sched/p4tc/trace.c       |  10 +
 net/sched/p4tc/trace.h       |  44 ++++
 7 files changed, 538 insertions(+), 1 deletion(-)
 create mode 100644 net/sched/cls_p4.c
 create mode 100644 net/sched/p4tc/trace.c
 create mode 100644 net/sched/p4tc/trace.h
include/uapi/linux/pkt_cls.h +18 -0
index dd313a727..e4d89bc98 100644
@@ -692,6 +692,24 @@ enum {
 
 #define TCA_MATCHALL_MAX (__TCA_MATCHALL_MAX - 1)
 
+/* P4 classifier */
+
+enum {
+	TCA_P4_UNSPEC,
+	TCA_P4_CLASSID,
+	TCA_P4_ACT,
+	TCA_P4_PNAME,
+	TCA_P4_PIPEID,
+	TCA_P4_PROG_FD,
+	TCA_P4_PROG_NAME,
+	TCA_P4_PROG_TYPE,
+	TCA_P4_PROG_ID,
+	TCA_P4_PAD,
+	__TCA_P4_MAX,
+};
+
+#define TCA_P4_MAX (__TCA_P4_MAX - 1)
+
 /* Extended Matches */
 
 struct tcf_ematch_tree_hdr {
net/sched/Kconfig +12 -0
index b1798a5e1..b33d790e1 100644
@@ -565,6 +565,18 @@ config NET_CLS_MATCHALL
 	  To compile this code as a module, choose M here: the module will
 	  be called cls_matchall.
 
+config NET_CLS_P4
+	tristate "P4 classifier"
+	select NET_CLS
+	select NET_P4TC
+	help
+	  If you say Y here, you will be able to bind a P4 pipeline
+	  program. You will need to install a P4 template representing the
+	  program successfully to use this feature.
+
+	  To compile this code as a module, choose M here: the module will
+	  be called cls_p4.
+
 config NET_EMATCH
 	bool "Extended Matches"
 	select NET_CLS
net/sched/Makefile +1 -0
index 581f9dd69..b4f9ef48d 100644
@@ -72,6 +72,7 @@ obj-$(CONFIG_NET_CLS_CGROUP)	+= cls_cgroup.o
 obj-$(CONFIG_NET_CLS_BPF)	+= cls_bpf.o
 obj-$(CONFIG_NET_CLS_FLOWER)	+= cls_flower.o
 obj-$(CONFIG_NET_CLS_MATCHALL)	+= cls_matchall.o
+obj-$(CONFIG_NET_CLS_P4)	+= cls_p4.o
 obj-$(CONFIG_NET_EMATCH)	+= ematch.o
 obj-$(CONFIG_NET_EMATCH_CMP)	+= em_cmp.o
 obj-$(CONFIG_NET_EMATCH_NBYTE)	+= em_nbyte.o
net/sched/cls_p4.c new file +450 -0

450 lines not on this page · raw message

net/sched/p4tc/Makefile +3 -1
index 73ccb53c4..04302a3ac 100644
@@ -1,6 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0
 
+CFLAGS_trace.o := -I$(src)
+
 obj-y := p4tc_types.o p4tc_tmpl_api.o p4tc_pipeline.o \
 	p4tc_action.o p4tc_table.o p4tc_tbl_entry.o \
-	p4tc_filter.o p4tc_runtime_api.o
+	p4tc_filter.o p4tc_runtime_api.o trace.o
 obj-$(CONFIG_DEBUG_INFO_BTF) += p4tc_bpf.o
net/sched/p4tc/trace.c new file +10 -0
new file mode 100644
index 000000000..683313407
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+#include <net/p4tc.h>
+
+#ifndef __CHECKER__
+
+#define CREATE_TRACE_POINTS
+#include "trace.h"
+EXPORT_TRACEPOINT_SYMBOL_GPL(p4_classify);
+#endif
net/sched/p4tc/trace.h new file +44 -0

44 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer Nacked-by on v10 15/15 NAK
#36 lore raw

Re: [PATCH v10 net-next 15/15] p4tc: add P4 classifier

Quoted 97 lines · Jamal Hadi Salim

Introduce P4 tc classifier. The main task of this classifier is to manage the lifetime of pipeline instances across one or more netdev ports. Note a pipeline may be instantiated multiple times across one or more tc chains and different priorities.

Note that part or whole of the P4 pipeline could reside in tc, XDP or even hardware depending on how the P4 program was compiled.
To use the P4 classifier you must specify a pipeline name that will be associated to the filter instance, a s/w parser (eBPF) and datapath P4 control block program (eBPF) program. Although this patchset does not deal with offloads, it is also possible to load the h/w part using this filter. We will illustrate a few examples further below to clarify. Please treat the illustrated split as an example - there are probably more pragmatic approaches to splitting the pipeline; however, regardless of where the different pieces of the pipeline are placed (tc, XDP, HW) and what each layer will implement (what part of the pipeline) - these examples are merely showing what is possible.

The pipeline is assumed to have already been created via a template.

For example, if we were to add a filter to ingress of a group of netdevs (tc block 22) and associate it to P4 pipeline simple_l3 we could issue the following command:

tc filter add block 22 parent ffff: protocol all prio 6 p4 pname simple_l3 \
action bpf obj $PARSER.o ... \
action bpf obj $PROGNAME.o section prog/tc-ingress

The above uses the classical tc action mechanism in which the first action runs the P4 parser and if that goes well then the P4 control block is executed. Note, although not shown above, one could also append the command line with other traditional tc actions.

In these patches, we also support two types of loadings of the pipeline programs and differentiate between what gets loaded at say tc vs xdp by using syntax which specifies location as either "prog type tc obj" or "prog type xdp obj". There is an ongoing discussion in the P4TC community biweekly meetings which is likely going to have us add another location definition "prog type hw" which will specify the hardware object file name and other related attributes.

An example using tc:

tc filter add block 22 parent ffff: protocol all prio 6 p4 pname simple_l3 \
prog type tc obj $PARSER.o ... \
action bpf obj $PROGNAME.o section prog/tc-ingress

For XDP, to illustrate an example:

tc filter add dev $P0 ingress protocol all prio 1 p4 pname simple_l3 \

     prog type xdp obj $PARSER.o section parser/xdp \
     pinned_link /sys/fs/bpf/mylink \
     action bpf obj $PROGNAME.o section prog/tc-ingress

In this case, the parser will be executed in the XDP layer and the rest of P4 control block as a tc action.

For illustration sake, the hw one looks as follows (please note there's still a lot of discussions going on in the meetings - the example is here merely to illustrate the tc filter functionality):

tc filter add block 22 ingress protocol all prio 1 p4 pname simple_l3 \

    prog type hw filename "mypnameprog.o" ... \
    prog type xdp obj $PARSER.o section parser/xdp pinned_link /sys/fs/bpf/mylink \
    action bpf obj $PROGNAME.o section prog/tc-ingress

The theory of operations is as follows:

================================1. PARSING================================

The packet first encounters the parser.
The parser is implemented in ebpf residing either at the TC or XDP level. The parsed header values are stored in a shared eBPF map. When the parser runs at XDP level, we load it into XDP using tc filter command and pin it to a file.

=============================2. ACTIONS=============================

In the above example, the P4 program (minus the parser) is encoded in an action($PROGNAME.o). It should be noted that classical tc actions continue to work:
IOW, someone could decide to add a mirred action to mirror all packets after or before the ebpf action.

tc filter add dev $P0 parent ffff: protocol all prio 6 p4 pname simple_l3 \

     prog type tc obj $PARSER.o section parser/tc-ingress \
     action bpf obj $PROGNAME.o section prog/tc-ingress \
     action mirred egress mirror index 1 dev $P1 \
     action bpf obj $ANOTHERPROG.o section mysect/section-1

It should also be noted that it is feasible to split some of the ingress datapath into XDP first and more into TC later (as was shown above for example where the parser runs at XDP level). YMMV. Regardless of choice of which scheme to use, none of these will affect UAPI. It will all depend on whether you generate code to load on XDP vs tc, etc.

Quoted 5 lines

My objections from last iterations still stand, and I also added a nak, so please do not just drop it with new revisions.. from the v10 as you wrote you added further code but despite the various community feedback the design still stands as before, therefore:

[...]

Quoted 4 lines
net/sched/cls_p4.c new file +4 -0
+static int cls_p4_prog_from_efd(struct nlattr **tb,
+				struct p4tc_bpf_prog *prog, u32 flags,
+				struct netlink_ext_ack *extack)
+{

net/sched/cls_p4.c line 176

Quoted 10 lines
net/sched/cls_p4.c new file +10 -0
+	struct bpf_prog *fp;
+	u32 prog_type;
+	char *name;
+	u32 bpf_fd;
+
+	bpf_fd = nla_get_u32(tb[TCA_P4_PROG_FD]);
+	prog_type = nla_get_u32(tb[TCA_P4_PROG_TYPE]);
+
+	if (prog_type != BPF_PROG_TYPE_XDP &&
+	    prog_type != BPF_PROG_TYPE_SCHED_ACT) {

Also as mentioned earlier I don't think tc should hold references on XDP programs in here. It doesn't make any sense aside from the fact that the cls_p4 is also not doing anything with it. This is something that a user space control plane should be doing i.e. managing a XDP link on the target device.

Quoted 21 lines
net/sched/cls_p4.c new file +21 -0
+		NL_SET_ERR_MSG(extack,
+			       "BPF prog type must be BPF_PROG_TYPE_SCHED_ACT or BPF_PROG_TYPE_XDP");
+		return -EINVAL;
+	}
+
+	fp = bpf_prog_get_type_dev(bpf_fd, prog_type, false);
+	if (IS_ERR(fp))
+		return PTR_ERR(fp);
+
+	name = nla_memdup(tb[TCA_P4_PROG_NAME], GFP_KERNEL);
+	if (!name) {
+		bpf_prog_put(fp);
+		return -ENOMEM;
+	}
+
+	prog->p4_prog_name = name;
+	prog->p4_prog = fp;
+
+	return 0;
+}
+
author maintainer
#37 lore raw

Re: [PATCH v10 net-next 15/15] p4tc: add P4 classifier

Quoted 105 lines · Daniel Borkmann
Quoted 97 lines · Jamal Hadi Salim

Introduce P4 tc classifier. The main task of this classifier is to manage the lifetime of pipeline instances across one or more netdev ports. Note a pipeline may be instantiated multiple times across one or more tc chains and different priorities.

Note that part or whole of the P4 pipeline could reside in tc, XDP or even hardware depending on how the P4 program was compiled.
To use the P4 classifier you must specify a pipeline name that will be associated to the filter instance, a s/w parser (eBPF) and datapath P4 control block program (eBPF) program. Although this patchset does not deal with offloads, it is also possible to load the h/w part using this filter. We will illustrate a few examples further below to clarify. Please treat the illustrated split as an example - there are probably more pragmatic approaches to splitting the pipeline; however, regardless of where the different pieces of the pipeline are placed (tc, XDP, HW) and what each layer will implement (what part of the pipeline) - these examples are merely showing what is possible.

The pipeline is assumed to have already been created via a template.

For example, if we were to add a filter to ingress of a group of netdevs (tc block 22) and associate it to P4 pipeline simple_l3 we could issue the following command:

tc filter add block 22 parent ffff: protocol all prio 6 p4 pname simple_l3 \
action bpf obj $PARSER.o ... \
action bpf obj $PROGNAME.o section prog/tc-ingress

The above uses the classical tc action mechanism in which the first action runs the P4 parser and if that goes well then the P4 control block is executed. Note, although not shown above, one could also append the command line with other traditional tc actions.

In these patches, we also support two types of loadings of the pipeline programs and differentiate between what gets loaded at say tc vs xdp by using syntax which specifies location as either "prog type tc obj" or "prog type xdp obj". There is an ongoing discussion in the P4TC community biweekly meetings which is likely going to have us add another location definition "prog type hw" which will specify the hardware object file name and other related attributes.

An example using tc:

tc filter add block 22 parent ffff: protocol all prio 6 p4 pname simple_l3 \
prog type tc obj $PARSER.o ... \
action bpf obj $PROGNAME.o section prog/tc-ingress

For XDP, to illustrate an example:

tc filter add dev $P0 ingress protocol all prio 1 p4 pname simple_l3 \

     prog type xdp obj $PARSER.o section parser/xdp \
     pinned_link /sys/fs/bpf/mylink \
     action bpf obj $PROGNAME.o section prog/tc-ingress

In this case, the parser will be executed in the XDP layer and the rest of P4 control block as a tc action.

For illustration sake, the hw one looks as follows (please note there's still a lot of discussions going on in the meetings - the example is here merely to illustrate the tc filter functionality):

tc filter add block 22 ingress protocol all prio 1 p4 pname simple_l3 \

    prog type hw filename "mypnameprog.o" ... \
    prog type xdp obj $PARSER.o section parser/xdp pinned_link /sys/fs/bpf/mylink \
    action bpf obj $PROGNAME.o section prog/tc-ingress

The theory of operations is as follows:

================================1. PARSING================================

The packet first encounters the parser.
The parser is implemented in ebpf residing either at the TC or XDP level. The parsed header values are stored in a shared eBPF map. When the parser runs at XDP level, we load it into XDP using tc filter command and pin it to a file.

=============================2. ACTIONS=============================

In the above example, the P4 program (minus the parser) is encoded in an action($PROGNAME.o). It should be noted that classical tc actions continue to work:
IOW, someone could decide to add a mirred action to mirror all packets after or before the ebpf action.

tc filter add dev $P0 parent ffff: protocol all prio 6 p4 pname simple_l3 \

     prog type tc obj $PARSER.o section parser/tc-ingress \
     action bpf obj $PROGNAME.o section prog/tc-ingress \
     action mirred egress mirror index 1 dev $P1 \
     action bpf obj $ANOTHERPROG.o section mysect/section-1

It should also be noted that it is feasible to split some of the ingress datapath into XDP first and more into TC later (as was shown above for example where the parser runs at XDP level). YMMV. Regardless of choice of which scheme to use, none of these will affect UAPI. It will all depend on whether you generate code to load on XDP vs tc, etc.

Quoted 5 lines
Quoted 7 lines

My objections from last iterations still stand, and I also added a nak, so please do not just drop it with new revisions.. from the v10 as you wrote you added further code but despite the various community feedback the design still stands as before, therefore:

We didnt make code changes - but did you read the cover letter and the extended commentary in this patch's commit log? We should have mentioned it in the changes log. It did respond to your comments. There's text that says "the filter manages the lifetime of the pipeline" - which in the future could include not only tc but XDP but also the hardware path (in the form of a file that gets loaded). I am not sure if that message is clear. Your angle being this is layer violation. In the last discussion i asked you for suggestions and we went the tcx route, which didnt make sense, and then you didnt respond.

Quoted 13 lines

[...]

Quoted 12 lines
net/sched/cls_p4.c new file +12 -0
+static int cls_p4_prog_from_efd(struct nlattr **tb,
+                             struct p4tc_bpf_prog *prog, u32 flags,
+                             struct netlink_ext_ack *extack)
+{
+     struct bpf_prog *fp;
+     u32 prog_type;
+     char *name;
+     u32 bpf_fd;
+
+     bpf_fd = nla_get_u32(tb[TCA_P4_PROG_FD]);
+     prog_type = nla_get_u32(tb[TCA_P4_PROG_TYPE]);
+
Quoted 8 lines

net/sched/cls_p4.c line 176

Quoted 2 lines
net/sched/cls_p4.c new file +2 -0
+     if (prog_type != BPF_PROG_TYPE_XDP &&
+         prog_type != BPF_PROG_TYPE_SCHED_ACT) {

Also as mentioned earlier I don't think tc should hold references on XDP programs in here. It doesn't make any sense aside from the fact that the cls_p4 is also not doing anything with it. This is something that a user space control plane should be doing i.e. managing a XDP link on the target device.

This is the same argument about layer violation that you made earlier. The filter manages the p4 pipeline - i.e it's not just about the ebpf blob(s) but for example in the future (discussions are still ongoing with vendors who have P4 NICs) a filter could be loaded to also specify the location of the hardware blob.
I would be happy with a suggestion that gets us moving forward with that context in mind.

cheers,
jamal

Quoted 21 lines
Quoted 21 lines
net/sched/cls_p4.c new file +21 -0
+             NL_SET_ERR_MSG(extack,
+                            "BPF prog type must be BPF_PROG_TYPE_SCHED_ACT or BPF_PROG_TYPE_XDP");
+             return -EINVAL;
+     }
+
+     fp = bpf_prog_get_type_dev(bpf_fd, prog_type, false);
+     if (IS_ERR(fp))
+             return PTR_ERR(fp);
+
+     name = nla_memdup(tb[TCA_P4_PROG_NAME], GFP_KERNEL);
+     if (!name) {
+             bpf_prog_put(fp);
+             return -ENOMEM;
+     }
+
+     prog->p4_prog_name = name;
+     prog->p4_prog = fp;
+
+     return 0;
+}
+
maintainer
#38 lore raw

Re: [PATCH v10 net-next 15/15] p4tc: add P4 classifier

Quoted 2 lines · Jamal Hadi Salim
Quoted 1 line · Daniel Borkmann

On 1/22/24 8:48 PM, Jamal Hadi Salim wrote:

[...]

Quoted 31 lines
Quoted 14 lines
Quoted 8 lines

It should also be noted that it is feasible to split some of the ingress datapath into XDP first and more into TC later (as was shown above for example where the parser runs at XDP level). YMMV. Regardless of choice of which scheme to use, none of these will affect UAPI. It will all depend on whether you generate code to load on XDP vs tc, etc.

Quoted 5 lines
Quoted 6 lines

My objections from last iterations still stand, and I also added a nak, so please do not just drop it with new revisions.. from the v10 as you wrote you added further code but despite the various community feedback the design still stands as before, therefore:

We didnt make code changes - but did you read the cover letter and the extended commentary in this patch's commit log? We should have mentioned it in the changes log. It did respond to your comments. There's text that says "the filter manages the lifetime of the pipeline" - which in the future could include not only tc but XDP but also the hardware path (in the form of a file that gets loaded). I am not sure if that message is clear. Your angle being this is layer violation. In the last discussion i asked you for suggestions and we went the tcx route, which didnt make sense, and then you didnt respond.

[...]

Quoted 11 lines
Quoted 5 lines

Also as mentioned earlier I don't think tc should hold references on XDP programs in here. It doesn't make any sense aside from the fact that the cls_p4 is also not doing anything with it. This is something that a user space control plane should be doing i.e. managing a XDP link on the target device.

This is the same argument about layer violation that you made earlier. The filter manages the p4 pipeline - i.e it's not just about the ebpf blob(s) but for example in the future (discussions are still ongoing with vendors who have P4 NICs) a filter could be loaded to also specify the location of the hardware blob.

Ah, so there is a plan to eventually add HW offload support for cls_p4? Or is this only specifiying a location of a blob through some opaque cookie value from user space?

Quoted 2 lines

I would be happy with a suggestion that gets us moving forward with that context in mind.

My question on the above is mainly what does it bring you to hold a reference on the XDP program? There is no guarantee that something else will get loaded onto XDP, and then eventually the cls_p4 is the only entity holding the reference but w/o 'purpose'. We do have BPF links and the user space component orchestrating all this needs to create and pin the BPF link in BPF fs, for example. An artificial reference on XDP prog feels similar as if you'd hold a reference on an inode out of tc.. Again, that should be delegated to the control plane you have running interacting with the compiler which then manages and loads its artifacts. What if you would also need to set up some netfilter rules for the SW pipeline, would you then embed this too?

Thanks,
Daniel

author maintainer
#39 lore raw

Re: [PATCH v10 net-next 15/15] p4tc: add P4 classifier

Quoted 50 lines · Daniel Borkmann
Quoted 2 lines · Jamal Hadi Salim
Quoted 1 line · Daniel Borkmann

On 1/22/24 8:48 PM, Jamal Hadi Salim wrote:

[...]

Quoted 31 lines
Quoted 14 lines
Quoted 8 lines

It should also be noted that it is feasible to split some of the ingress datapath into XDP first and more into TC later (as was shown above for example where the parser runs at XDP level). YMMV. Regardless of choice of which scheme to use, none of these will affect UAPI. It will all depend on whether you generate code to load on XDP vs tc, etc.

Quoted 5 lines
Quoted 6 lines

My objections from last iterations still stand, and I also added a nak, so please do not just drop it with new revisions.. from the v10 as you wrote you added further code but despite the various community feedback the design still stands as before, therefore:

We didnt make code changes - but did you read the cover letter and the extended commentary in this patch's commit log? We should have mentioned it in the changes log. It did respond to your comments. There's text that says "the filter manages the lifetime of the pipeline" - which in the future could include not only tc but XDP but also the hardware path (in the form of a file that gets loaded). I am not sure if that message is clear. Your angle being this is layer violation. In the last discussion i asked you for suggestions and we went the tcx route, which didnt make sense, and then you didnt respond.

[...]

Quoted 11 lines
Quoted 5 lines

Also as mentioned earlier I don't think tc should hold references on XDP programs in here. It doesn't make any sense aside from the fact that the cls_p4 is also not doing anything with it. This is something that a user space control plane should be doing i.e. managing a XDP link on the target device.

This is the same argument about layer violation that you made earlier. The filter manages the p4 pipeline - i.e it's not just about the ebpf blob(s) but for example in the future (discussions are still ongoing with vendors who have P4 NICs) a filter could be loaded to also specify the location of the hardware blob.

Quoted 3 lines

Ah, so there is a plan to eventually add HW offload support for cls_p4? Or is this only specifiying a location of a blob through some opaque cookie value from user space?

Current thought process is it will be something along these lines (the commit provides more details):

tc filter add block 22 ingress protocol all prio 1 p4 pname simple_l3 \
prog type hw filename "mypnameprog.o" ... \
prog type xdp obj $PARSER.o section parser/xdp pinned_link
/sys/fs/bpf/mylink \
action bpf obj $PROGNAME.o section prog/tc-ingress

These discussions are still ongoing - but that is the current consensus. Note: we are not pushing any code for that, but hope it paints the bigger picture....
The idea is the cls p4 owns the lifetime of the pipeline. Installing the filter instantiates the p4 pipeline "simple_l3" and triggers a lot of the refcounts to make sure the pipeline and its components stays alive.
There could be multiple such filters - when someone deletes the last filter, then it is safe to delete the pipeline.
Essentially the filter manages the lifetime of the pipeline.

Quoted 14 lines
Quoted 2 lines

I would be happy with a suggestion that gets us moving forward with that context in mind.

My question on the above is mainly what does it bring you to hold a reference on the XDP program? There is no guarantee that something else will get loaded onto XDP, and then eventually the cls_p4 is the only entity holding the reference but w/o 'purpose'. We do have BPF links and the user space component orchestrating all this needs to create and pin the BPF link in BPF fs, for example. An artificial reference on XDP prog feels similar as if you'd hold a reference on an inode out of tc.. Again, that should be delegated to the control plane you have running interacting with the compiler which then manages and loads its artifacts. What if you would also need to set up some netfilter rules for the SW pipeline, would you then embed this too?

Sorry, a slight tangent first:
P4 is self-contained, there are a handful of objects that are defined by the spec (externs, actions, tables, etc) and we model them in the patchset, so that part is self-contained. For the extra richness such as the netfilter example you quoted - based on my many years of experience deploying SDN - using daemons(sorry if i am reading too much in what I think you are implying) for control is not the best option i.e you need all kinds of coordination - for example where do you store state, what happens when the daemon dies, how do you graceful restarts etc. Based on that, if i can put things in the kernel (which is essentially a "perpetual daemon", unless the kernel crashes) it's a lot simpler to manage as a source of truth especially when there is not that much info. There is a limit when there are multiple pieces (to use your netfilter example) because you need another layer to coordinate things.

Re: the XDP part - our key reason is mostly managerial, in that the filter is the lifetime manager of the pipeline; and that if i dump that filter i can see all the details in regards to the pipeline(tc, XDP and in future hw, etc) in one spot. You are right, the link pinning is our protection from someone replacing the XDP prog (this was a tip from Toke in the early days) and the comparison of tc holding inode is apropos.
There's some history: in the early days we were also using metadata which comes from the XDP program at the tc layer if more processing was to be done (and there was extra metadata which told us which XDP prog produced it which we would vet before trusting the metadata). Given all the above, we should still be able to hold this info without necessarily holding the extra refcount and be able to see this detail. So we can remove the refcounting.

cheers,
jamal

Quoted 2 lines

Thanks,
Daniel

author maintainer
#40 lore raw

Re: [PATCH v10 net-next 15/15] p4tc: add P4 classifier

Quoted 107 lines · Jamal Hadi Salim
Quoted 50 lines · Daniel Borkmann
Quoted 2 lines · Jamal Hadi Salim
Quoted 1 line · Daniel Borkmann

On 1/22/24 8:48 PM, Jamal Hadi Salim wrote:

[...]

Quoted 31 lines
Quoted 14 lines
Quoted 8 lines

It should also be noted that it is feasible to split some of the ingress datapath into XDP first and more into TC later (as was shown above for example where the parser runs at XDP level). YMMV. Regardless of choice of which scheme to use, none of these will affect UAPI. It will all depend on whether you generate code to load on XDP vs tc, etc.

Quoted 5 lines
Quoted 6 lines

My objections from last iterations still stand, and I also added a nak, so please do not just drop it with new revisions.. from the v10 as you wrote you added further code but despite the various community feedback the design still stands as before, therefore:

We didnt make code changes - but did you read the cover letter and the extended commentary in this patch's commit log? We should have mentioned it in the changes log. It did respond to your comments. There's text that says "the filter manages the lifetime of the pipeline" - which in the future could include not only tc but XDP but also the hardware path (in the form of a file that gets loaded). I am not sure if that message is clear. Your angle being this is layer violation. In the last discussion i asked you for suggestions and we went the tcx route, which didnt make sense, and then you didnt respond.

[...]

Quoted 11 lines
Quoted 5 lines

Also as mentioned earlier I don't think tc should hold references on XDP programs in here. It doesn't make any sense aside from the fact that the cls_p4 is also not doing anything with it. This is something that a user space control plane should be doing i.e. managing a XDP link on the target device.

This is the same argument about layer violation that you made earlier. The filter manages the p4 pipeline - i.e it's not just about the ebpf blob(s) but for example in the future (discussions are still ongoing with vendors who have P4 NICs) a filter could be loaded to also specify the location of the hardware blob.

Quoted 3 lines

Ah, so there is a plan to eventually add HW offload support for cls_p4? Or is this only specifiying a location of a blob through some opaque cookie value from user space?

Current thought process is it will be something along these lines (the commit provides more details):

tc filter add block 22 ingress protocol all prio 1 p4 pname simple_l3 \
prog type hw filename "mypnameprog.o" ... \
prog type xdp obj $PARSER.o section parser/xdp pinned_link
/sys/fs/bpf/mylink \
action bpf obj $PROGNAME.o section prog/tc-ingress

These discussions are still ongoing - but that is the current consensus. Note: we are not pushing any code for that, but hope it paints the bigger picture....
The idea is the cls p4 owns the lifetime of the pipeline. Installing the filter instantiates the p4 pipeline "simple_l3" and triggers a lot of the refcounts to make sure the pipeline and its components stays alive.
There could be multiple such filters - when someone deletes the last filter, then it is safe to delete the pipeline.
Essentially the filter manages the lifetime of the pipeline.

Quoted 14 lines
Quoted 2 lines

I would be happy with a suggestion that gets us moving forward with that context in mind.

My question on the above is mainly what does it bring you to hold a reference on the XDP program? There is no guarantee that something else will get loaded onto XDP, and then eventually the cls_p4 is the only entity holding the reference but w/o 'purpose'. We do have BPF links and the user space component orchestrating all this needs to create and pin the BPF link in BPF fs, for example. An artificial reference on XDP prog feels similar as if you'd hold a reference on an inode out of tc.. Again, that should be delegated to the control plane you have running interacting with the compiler which then manages and loads its artifacts. What if you would also need to set up some netfilter rules for the SW pipeline, would you then embed this too?

Sorry, a slight tangent first:
P4 is self-contained, there are a handful of objects that are defined by the spec (externs, actions, tables, etc) and we model them in the patchset, so that part is self-contained. For the extra richness such as the netfilter example you quoted - based on my many years of experience deploying SDN - using daemons(sorry if i am reading too much in what I think you are implying) for control is not the best option i.e you need all kinds of coordination - for example where do you store state, what happens when the daemon dies, how do you graceful restarts etc. Based on that, if i can put things in the kernel (which is essentially a "perpetual daemon", unless the kernel crashes) it's a lot simpler to manage as a source of truth especially when there is not that much info. There is a limit when there are multiple pieces (to use your netfilter example) because you need another layer to coordinate things.

Quoted 15 lines

Re: the XDP part - our key reason is mostly managerial, in that the filter is the lifetime manager of the pipeline; and that if i dump that filter i can see all the details in regards to the pipeline(tc, XDP and in future hw, etc) in one spot. You are right, the link pinning is our protection from someone replacing the XDP prog (this was a tip from Toke in the early days) and the comparison of tc holding inode is apropos.
There's some history: in the early days we were also using metadata which comes from the XDP program at the tc layer if more processing was to be done (and there was extra metadata which told us which XDP prog produced it which we would vet before trusting the metadata). Given all the above, we should still be able to hold this info without necessarily holding the extra refcount and be able to see this detail. So we can remove the refcounting.

Daniel?

cheers,
jamal

Quoted 5 lines

cheers,
jamal

Quoted 2 lines

Thanks,
Daniel

maintainer
#41 lore raw

Re: [PATCH v10 net-next 15/15] p4tc: add P4 classifier

Quoted 90 lines · Jamal Hadi Salim
Quoted 89 lines · Jamal Hadi Salim
Quoted 49 lines · Daniel Borkmann
Quoted 2 lines · Jamal Hadi Salim
Quoted 1 line · Daniel Borkmann

On 1/22/24 8:48 PM, Jamal Hadi Salim wrote:

[...]

Quoted 31 lines
Quoted 14 lines
Quoted 8 lines

It should also be noted that it is feasible to split some of the ingress datapath into XDP first and more into TC later (as was shown above for example where the parser runs at XDP level). YMMV. Regardless of choice of which scheme to use, none of these will affect UAPI. It will all depend on whether you generate code to load on XDP vs tc, etc.

Quoted 5 lines
Quoted 6 lines

My objections from last iterations still stand, and I also added a nak, so please do not just drop it with new revisions.. from the v10 as you wrote you added further code but despite the various community feedback the design still stands as before, therefore:

We didnt make code changes - but did you read the cover letter and the extended commentary in this patch's commit log? We should have mentioned it in the changes log. It did respond to your comments. There's text that says "the filter manages the lifetime of the pipeline" - which in the future could include not only tc but XDP but also the hardware path (in the form of a file that gets loaded). I am not sure if that message is clear. Your angle being this is layer violation. In the last discussion i asked you for suggestions and we went the tcx route, which didnt make sense, and then you didnt respond.

[...]

Quoted 11 lines
Quoted 5 lines

Also as mentioned earlier I don't think tc should hold references on XDP programs in here. It doesn't make any sense aside from the fact that the cls_p4 is also not doing anything with it. This is something that a user space control plane should be doing i.e. managing a XDP link on the target device.

This is the same argument about layer violation that you made earlier. The filter manages the p4 pipeline - i.e it's not just about the ebpf blob(s) but for example in the future (discussions are still ongoing with vendors who have P4 NICs) a filter could be loaded to also specify the location of the hardware blob.

Quoted 3 lines

Ah, so there is a plan to eventually add HW offload support for cls_p4? Or is this only specifiying a location of a blob through some opaque cookie value from user space?

Current thought process is it will be something along these lines (the commit provides more details):

tc filter add block 22 ingress protocol all prio 1 p4 pname simple_l3 \
prog type hw filename "mypnameprog.o" ... \
prog type xdp obj $PARSER.o section parser/xdp pinned_link
/sys/fs/bpf/mylink \
action bpf obj $PROGNAME.o section prog/tc-ingress

These discussions are still ongoing - but that is the current consensus. Note: we are not pushing any code for that, but hope it paints the bigger picture....
The idea is the cls p4 owns the lifetime of the pipeline. Installing the filter instantiates the p4 pipeline "simple_l3" and triggers a lot of the refcounts to make sure the pipeline and its components stays alive.
There could be multiple such filters - when someone deletes the last filter, then it is safe to delete the pipeline.
Essentially the filter manages the lifetime of the pipeline.

Quoted 14 lines
Quoted 2 lines

I would be happy with a suggestion that gets us moving forward with that context in mind.

My question on the above is mainly what does it bring you to hold a reference on the XDP program? There is no guarantee that something else will get loaded onto XDP, and then eventually the cls_p4 is the only entity holding the reference but w/o 'purpose'. We do have BPF links and the user space component orchestrating all this needs to create and pin the BPF link in BPF fs, for example. An artificial reference on XDP prog feels similar as if you'd hold a reference on an inode out of tc.. Again, that should be delegated to the control plane you have running interacting with the compiler which then manages and loads its artifacts. What if you would also need to set up some netfilter rules for the SW pipeline, would you then embed this too?

Quoted 15 lines
Quoted 7 lines

Sorry, a slight tangent first:
P4 is self-contained, there are a handful of objects that are defined by the spec (externs, actions, tables, etc) and we model them in the patchset, so that part is self-contained. For the extra richness such as the netfilter example you quoted - based on my many years of experience deploying SDN - using daemons(sorry if i am reading too much in what I think you are implying) for control is not the best

Quoted 8 lines

option i.e you need all kinds of coordination - for example where do you store state, what happens when the daemon dies, how do you graceful restarts etc. Based on that, if i can put things in the kernel (which is essentially a "perpetual daemon", unless the kernel crashes) it's a lot simpler to manage as a source of truth especially when there is not that much info. There is a limit when there are multiple pieces (to use your netfilter example) because you need another layer to coordinate things.

'source of truth' for the various attach points or BPF links, yes, but in this case here it is not, since the source of truth on what is attached is not in cls_p4 but rather on the XDP link. How do you handle the case when cls_p4 says something different to what is /actually/ attached? Why is it not enough to establish some convention in user space, to pin the link and retrieve/update from there when needed? Like everyone else does. ... even if you consider iproute2 your "control plane" (which I have the feeling you do)?

Quoted 2 lines
Quoted 2 lines

Re: the XDP part - our key reason is mostly managerial, in that the filter is the lifetime manager of the pipeline; and that if i dump

This is imho the problematic part which feels like square peg in round hole, trying to fit this whole lifetime manager of the pipeline into the cls_p4 filter. We agree to disagree here. Instead of reusing individual building blocks from user space, this tries to cramp control plane parts into the kernel for which its not a great fit with what is build here as-is.

Quoted 13 lines
Quoted 4 lines

that filter i can see all the details in regards to the pipeline(tc, XDP and in future hw, etc) in one spot. You are right, the link pinning is our protection from someone replacing the XDP prog (this was a tip from Toke in the early days) and the comparison of tc

Quoted 8 lines

holding inode is apropos.
There's some history: in the early days we were also using metadata which comes from the XDP program at the tc layer if more processing was to be done (and there was extra metadata which told us which XDP prog produced it which we would vet before trusting the metadata). Given all the above, we should still be able to hold this info without necessarily holding the extra refcount and be able to see this detail. So we can remove the refcounting.

Quoted 1 line

Daniel?

The refcount should definitely be removed, but then again, see the point above in that it is inconsistent information. Why can't this be done in user space with some convention in your user space control plane - if you take iproute2, then why it cannot pin the link in a bpf fs instance and retrieve it from there?

author maintainer
#42 lore raw

Re: [PATCH v10 net-next 15/15] p4tc: add P4 classifier

Quoted 140 lines · Daniel Borkmann
Quoted 90 lines · Jamal Hadi Salim
Quoted 89 lines · Jamal Hadi Salim
Quoted 49 lines · Daniel Borkmann
Quoted 2 lines · Jamal Hadi Salim
Quoted 1 line · Daniel Borkmann

On 1/22/24 8:48 PM, Jamal Hadi Salim wrote:

[...]

Quoted 31 lines
Quoted 14 lines
Quoted 8 lines

It should also be noted that it is feasible to split some of the ingress datapath into XDP first and more into TC later (as was shown above for example where the parser runs at XDP level). YMMV. Regardless of choice of which scheme to use, none of these will affect UAPI. It will all depend on whether you generate code to load on XDP vs tc, etc.

Quoted 5 lines
Quoted 6 lines

My objections from last iterations still stand, and I also added a nak, so please do not just drop it with new revisions.. from the v10 as you wrote you added further code but despite the various community feedback the design still stands as before, therefore:

We didnt make code changes - but did you read the cover letter and the extended commentary in this patch's commit log? We should have mentioned it in the changes log. It did respond to your comments. There's text that says "the filter manages the lifetime of the pipeline" - which in the future could include not only tc but XDP but also the hardware path (in the form of a file that gets loaded). I am not sure if that message is clear. Your angle being this is layer violation. In the last discussion i asked you for suggestions and we went the tcx route, which didnt make sense, and then you didnt respond.

[...]

Quoted 11 lines
Quoted 5 lines

Also as mentioned earlier I don't think tc should hold references on XDP programs in here. It doesn't make any sense aside from the fact that the cls_p4 is also not doing anything with it. This is something that a user space control plane should be doing i.e. managing a XDP link on the target device.

This is the same argument about layer violation that you made earlier. The filter manages the p4 pipeline - i.e it's not just about the ebpf blob(s) but for example in the future (discussions are still ongoing with vendors who have P4 NICs) a filter could be loaded to also specify the location of the hardware blob.

Quoted 3 lines

Ah, so there is a plan to eventually add HW offload support for cls_p4? Or is this only specifiying a location of a blob through some opaque cookie value from user space?

Current thought process is it will be something along these lines (the commit provides more details):

tc filter add block 22 ingress protocol all prio 1 p4 pname simple_l3 \
prog type hw filename "mypnameprog.o" ... \
prog type xdp obj $PARSER.o section parser/xdp pinned_link
/sys/fs/bpf/mylink \
action bpf obj $PROGNAME.o section prog/tc-ingress

These discussions are still ongoing - but that is the current consensus. Note: we are not pushing any code for that, but hope it paints the bigger picture....
The idea is the cls p4 owns the lifetime of the pipeline. Installing the filter instantiates the p4 pipeline "simple_l3" and triggers a lot of the refcounts to make sure the pipeline and its components stays alive.
There could be multiple such filters - when someone deletes the last filter, then it is safe to delete the pipeline.
Essentially the filter manages the lifetime of the pipeline.

Quoted 14 lines
Quoted 2 lines

I would be happy with a suggestion that gets us moving forward with that context in mind.

My question on the above is mainly what does it bring you to hold a reference on the XDP program? There is no guarantee that something else will get loaded onto XDP, and then eventually the cls_p4 is the only entity holding the reference but w/o 'purpose'. We do have BPF links and the user space component orchestrating all this needs to create and pin the BPF link in BPF fs, for example. An artificial reference on XDP prog feels similar as if you'd hold a reference on an inode out of tc.. Again, that should be delegated to the control plane you have running interacting with the compiler which then manages and loads its artifacts. What if you would also need to set up some netfilter rules for the SW pipeline, would you then embed this too?

Quoted 15 lines
Quoted 7 lines

Sorry, a slight tangent first:
P4 is self-contained, there are a handful of objects that are defined by the spec (externs, actions, tables, etc) and we model them in the patchset, so that part is self-contained. For the extra richness such as the netfilter example you quoted - based on my many years of experience deploying SDN - using daemons(sorry if i am reading too much in what I think you are implying) for control is not the best

Quoted 8 lines

option i.e you need all kinds of coordination - for example where do you store state, what happens when the daemon dies, how do you graceful restarts etc. Based on that, if i can put things in the kernel (which is essentially a "perpetual daemon", unless the kernel crashes) it's a lot simpler to manage as a source of truth especially when there is not that much info. There is a limit when there are multiple pieces (to use your netfilter example) because you need another layer to coordinate things.

'source of truth' for the various attach points or BPF links, yes, but in this case here it is not, since the source of truth on what is attached is not in cls_p4 but rather on the XDP link. How do you handle the case when cls_p4 says something different to what is /actually/ attached? Why is it not enough to establish some convention in user space, to pin the link and retrieve/update from there when needed? Like everyone else does. ... even if you consider iproute2 your "control plane" (which I have the feeling you do)?

Quoted 2 lines
Quoted 2 lines

Re: the XDP part - our key reason is mostly managerial, in that the filter is the lifetime manager of the pipeline; and that if i dump

This is imho the problematic part which feels like square peg in round hole, trying to fit this whole lifetime manager of the pipeline into the cls_p4 filter. We agree to disagree here. Instead of reusing individual building blocks from user space, this tries to cramp control plane parts into the kernel for which its not a great fit with what is build here as-is.

Quoted 13 lines
Quoted 12 lines

that filter i can see all the details in regards to the pipeline(tc, XDP and in future hw, etc) in one spot. You are right, the link pinning is our protection from someone replacing the XDP prog (this was a tip from Toke in the early days) and the comparison of tc holding inode is apropos.
There's some history: in the early days we were also using metadata which comes from the XDP program at the tc layer if more processing was to be done (and there was extra metadata which told us which XDP prog produced it which we would vet before trusting the metadata). Given all the above, we should still be able to hold this info without necessarily holding the extra refcount and be able to see this detail. So we can remove the refcounting.

Quoted 7 lines
Quoted 1 line

Daniel?

The refcount should definitely be removed, but then again, see the point above in that it is inconsistent information. Why can't this be done in user space with some convention in your user space control plane - if you take iproute2, then why it cannot pin the link in a bpf fs instance and retrieve it from there?

Ok, Daniel - let's do this so we can move forward. I am getting exhausted, we've been going at this for a year now. As a compromise: I will remove the support for XDP altogether from the filter. We will still reference the XDP program in the CLI and infact load and pin it that way but the filter will not be adding a refcount in the kernel as in the posted patch.

cheers,
jamal