Skip to contentSkip to conversation

build: check drivers class dependencies early

PATCH Superseded David Marchand 4 participants 11 messages Aug 1, 2023 – Aug 8, 2023 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 build: check drivers class dependencies early

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

---
 drivers/meson.build | 11 +++++++++++
 1 file changed, 11 insertions(+)
drivers/meson.build +11 -0
index 74ae8cb96b..c375352e77 100644
@@ -70,6 +70,17 @@ foreach subpath:subdirs
     else
         class = subpath
         subdir(class)
+        skip_class = false
+        foreach d:std_deps
+            if not is_variable('shared_rte_' + d)
+                skip_class = true
+                message('Disabling all @1@ drivers: missing internal dependency "@0@"'
+                        .format(d, class))
+            endif
+        endforeach
+        if skip_class
+            continue
+        endif
     endif
 
     # save class name on first occurrence
Signature · 1 line
2.41.0
#1 lore raw
Quoted 29 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

---
 drivers/meson.build | 11 +++++++++++
 1 file changed, 11 insertions(+)
drivers/meson.build +4 -0
index 74ae8cb96b..c375352e77 100644
@@ -70,6 +70,17 @@ foreach subpath:subdirs
     else
         class = subpath
         subdir(class)
+        skip_class = false
+        foreach d:std_deps
+            if not is_variable('shared_rte_' + d)
+                skip_class = true

drivers/meson.build foreach subpath:subdirs · line 86

Quoted 10 lines
drivers/meson.build +7 -0
+                message('Disabling all @1@ drivers: missing internal dependency "@0@"'
+                        .format(d, class))
+            endif
+        endforeach
+        if skip_class
+            continue
+        endif
     endif
 
     # save class name on first occurrence

I like this approach. However, we do need something in the summary at the end of the build too. Either:

* Single message stating all drivers of a given class are skipped
* (as now), message for each driver stating that it has been disabled

The former is nice as it gives us a shorter summary. The latter is nice because it's consistent with what we have now. Authors choice, which to go for! :-)

Thanks.
/Bruce

#2 lore raw
Quoted 45 lines · Bruce Richardson
Quoted 29 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

---
 drivers/meson.build | 11 +++++++++++
 1 file changed, 11 insertions(+)
drivers/meson.build +4 -0
index 74ae8cb96b..c375352e77 100644
@@ -70,6 +70,17 @@ foreach subpath:subdirs
     else
         class = subpath
         subdir(class)
+        skip_class = false
+        foreach d:std_deps
+            if not is_variable('shared_rte_' + d)
+                skip_class = true

drivers/meson.build foreach subpath:subdirs · line 86

Quoted 10 lines
drivers/meson.build +7 -0
+                message('Disabling all @1@ drivers: missing internal dependency "@0@"'
+                        .format(d, class))
+            endif
+        endforeach
+        if skip_class
+            continue
+        endif
     endif
 
     # save class name on first occurrence

I like this approach. However, we do need something in the summary at the end of the build too. Either:

Quoted 6 lines

* Single message stating all drivers of a given class are skipped
* (as now), message for each driver stating that it has been disabled

The former is nice as it gives us a shorter summary. The latter is nice because it's consistent with what we have now. Authors choice, which to go for! :-)

I like the shorter summary.
WDYT of:

drivers/meson.build +0 -1
index c375352e77..02268918e4 100644
@@ -74,11 +74,18 @@ foreach subpath:subdirs
         foreach d:std_deps
             if not is_variable('shared_rte_' + d)
                 skip_class = true
-                message('Disabling all @1@ drivers: missing internal

dependency "@0@"'
+ reason = 'missing internal dependency, "@0@"'.format(d)
+ if dpdk_libs_deprecated.contains(d)
+ reason += ' (deprecated lib)'
+ endif
+ message('Disabling @1@/* drivers: missing internal
dependency "@0@"'

                         .format(d, class))
             endif
         endforeach
         if skip_class
+            drv_path = join_paths(class, '*')
+            dpdk_drvs_disabled += drv_path
+            set_variable(drv_path.underscorify() + '_disable_reason', reason)

continue
endif
endif

This gives the following output (testing on top of your series extending optional libs and a fix on the new reasm perf test).
https://github.com/david-marchand/dpdk/actions/runs/5725429526/job/15513923381#step:19:346

...
=================
Content Skipped
=================
...
drivers:
...
mempool/dpaa2: not in enabled drivers build config mempool/octeontx: not in enabled drivers build config mempool/stack: not in enabled drivers build config dma/*: missing internal dependency, "dmadev" net/af_packet: not in enabled drivers build config net/af_xdp: not in enabled drivers build config net/ark: not in enabled drivers build config
...

Signature · 1 line
David Marchand
#3 lore raw
Quoted 73 lines · David Marchand
Quoted 45 lines · Bruce Richardson
Quoted 29 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

---
 drivers/meson.build | 11 +++++++++++
 1 file changed, 11 insertions(+)
drivers/meson.build +4 -0
index 74ae8cb96b..c375352e77 100644
@@ -70,6 +70,17 @@ foreach subpath:subdirs
     else
         class = subpath
         subdir(class)
+        skip_class = false
+        foreach d:std_deps
+            if not is_variable('shared_rte_' + d)
+                skip_class = true

drivers/meson.build foreach subpath:subdirs · line 86

Quoted 10 lines
drivers/meson.build +7 -0
+                message('Disabling all @1@ drivers: missing internal dependency "@0@"'
+                        .format(d, class))
+            endif
+        endforeach
+        if skip_class
+            continue
+        endif
     endif
 
     # save class name on first occurrence

I like this approach. However, we do need something in the summary at the end of the build too. Either:

Quoted 6 lines

* Single message stating all drivers of a given class are skipped
* (as now), message for each driver stating that it has been disabled

The former is nice as it gives us a shorter summary. The latter is nice because it's consistent with what we have now. Authors choice, which to go for! :-)

I like the shorter summary.
WDYT of:

drivers/meson.build +0 -1
index c375352e77..02268918e4 100644
@@ -74,11 +74,18 @@ foreach subpath:subdirs
         foreach d:std_deps
             if not is_variable('shared_rte_' + d)
                 skip_class = true
-                message('Disabling all @1@ drivers: missing internal

dependency "@0@"'

drivers/meson.build +5 -0
+                reason = 'missing internal dependency, "@0@"'.format(d)
+                if dpdk_libs_deprecated.contains(d)
+                    reason += ' (deprecated lib)'
+                endif
+                message('Disabling @1@/* drivers: missing internal

dependency "@0@"'

Quoted 1 line
                         .format(d, class))

Minor nit: You probably want a break here, as we only have room for one missing dependency, so no point in checking more after the first missing.

Quoted 15 lines

endif
endforeach
if skip_class

drivers/meson.build +3 -0
+            drv_path = join_paths(class, '*')
+            dpdk_drvs_disabled += drv_path
+            set_variable(drv_path.underscorify() + '_disable_reason', reason)
             continue
         endif
     endif

This gives the following output (testing on top of your series extending optional libs and a fix on the new reasm perf test).
https://github.com/david-marchand/dpdk/actions/runs/5725429526/job/15513923381#step:19:346

Quoted 16 lines

...
=================
Content Skipped
=================
...
drivers:
...
mempool/dpaa2: not in enabled drivers build config mempool/octeontx: not in enabled drivers build config mempool/stack: not in enabled drivers build config dma/*: missing internal dependency, "dmadev" net/af_packet: not in enabled drivers build config net/af_xdp: not in enabled drivers build config net/ark: not in enabled drivers build config
...

+1 to this. Works for me.

#4 lore raw
In reply to David Marchand
Quoted 27 lines

From: David Marchand [mailto:[email protected]]
Sent: Tuesday, 1 August 2023 12.21

Quoted 10 lines · Bruce Richardson
Quoted 8 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

Quoted 6 lines
Quoted 6 lines

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

---

This gives the following output (testing on top of your series extending optional libs and a fix on the new reasm perf test).
https://github.com/david-
marchand/dpdk/actions/runs/5725429526/job/15513923381#step:19:346

Quoted 16 lines

...
=================
Content Skipped
=================
...
drivers:
...
mempool/dpaa2: not in enabled drivers build config mempool/octeontx: not in enabled drivers build config mempool/stack: not in enabled drivers build config dma/*: missing internal dependency, "dmadev" net/af_packet: not in enabled drivers build config net/af_xdp: not in enabled drivers build config net/ark: not in enabled drivers build config
...

For the concept suggested by David (and preferably with Bruce's output):

#5 lore raw
Quoted 49 lines

From: Morten Brørup [mailto:[email protected]]
Sent: Tuesday, 1 August 2023 12.49

Quoted 14 lines

From: David Marchand [mailto:[email protected]]
Sent: Tuesday, 1 August 2023 12.21

Quoted 9 lines · Bruce Richardson
Quoted 7 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may

break.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build

when

Quoted 10 lines
Quoted 4 lines
Quoted 4 lines

the class dependencies are not met.

---

This gives the following output (testing on top of your series extending optional libs and a fix on the new reasm perf test).
https://github.com/david-
marchand/dpdk/actions/runs/5725429526/job/15513923381#step:19:346

Quoted 16 lines

...
=================
Content Skipped
=================
...
drivers:
...
mempool/dpaa2: not in enabled drivers build config mempool/octeontx: not in enabled drivers build config mempool/stack: not in enabled drivers build config dma/*: missing internal dependency, "dmadev" net/af_packet: not in enabled drivers build config net/af_xdp: not in enabled drivers build config net/ark: not in enabled drivers build config
...

Quoted 1 line

For the concept suggested by David (and preferably with Bruce's output):

Sorry... This was David's suggestion, which I prefer. I thought I was replying to Bruce's reply.

Quoted 2 lines

My ACK still valid.

#6 lore raw

PATCH v2 1/1 build: check drivers class dependencies early

In reply to David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

---
Changes since v1:
- added break on the first missing dependency,
- added logs for disabled drivers,

---
 drivers/meson.build | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
drivers/meson.build +19 -0
index 74ae8cb96b..c909070c30 100644
@@ -70,6 +70,25 @@ foreach subpath:subdirs
     else
         class = subpath
         subdir(class)
+        skip_class = false
+        foreach d:std_deps
+            if not is_variable('shared_rte_' + d)
+                skip_class = true
+                reason = 'missing internal dependency, "@0@"'.format(d)
+                if dpdk_libs_deprecated.contains(d)
+                    reason += ' (deprecated lib)'
+                endif
+                message('Disabling @1@/* drivers: missing internal dependency "@0@"'
+                        .format(d, class))
+                break
+            endif
+        endforeach
+        if skip_class
+            drv_path = join_paths(class, '*')
+            dpdk_drvs_disabled += drv_path
+            set_variable(drv_path.underscorify() + '_disable_reason', reason)
+            continue
+        endif
     endif
 
     # save class name on first occurrence
Signature · 1 line
2.41.0
Acked-by on v2 1/1
#7 lore raw
Quoted 8 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

Quoted 6 lines

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

#8 lore raw
Quoted 9 lines · Bruce Richardson
Quoted 8 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

Quoted 7 lines
Quoted 6 lines

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

author Acked-by Bruce Richardson Acked-by Tyler Retzlaff on v2 1/1 Applied
#9 lore raw
In reply to David Marchand
Quoted 9 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

Quoted 6 lines

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

Applied, thanks.

Bruce, could you rebase your series marking more libraries optional? There is some conflicts after removal of kni/flow_classify, and adding a unit test (perf reasm).

Signature · 1 line
David Marchand
#10 lore raw
Quoted 16 lines · David Marchand
Quoted 9 lines · David Marchand

Drivers implementing a class of devices (for example, drivers/event) depend on the associated abstraction library (lib/eventdev). This dependency is expressed in the top level meson.build for this class (drivers/event/meson.build).

As we are making more libraries optional, custom constructs referencing the class dependencies in some drivers meson.build (event/dlb2) may break.

Quoted 6 lines

It would be possible to add more checks in those drivers meson.build but it is more straightforward to not even consider a driver meson.build when the class dependencies are not met.

Quoted 9 lines

Applied, thanks.

Bruce, could you rebase your series marking more libraries optional? There is some conflicts after removal of kni/flow_classify, and adding a unit test (perf reasm).

Yes, I plan to do so. However, I think I'd rather defer that set until after the set reworking the build of the tests completely. It makes everything a lot simpler in that regard. In fact, I may merge the two sets into one.

/Bruce