Skip to contentSkip to conversation

External app builds need to locate common make fragments and includes.

PATCH In review Keith Wiles 2 participants 8 messages Feb 28, 2015 – Mar 4, 2015 6 messages not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 External app builds need to locate common make fragments and includes.

When building an external application like Pktgen and using the proper makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk files are missing some defines/includes.

1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose ouput
as the Q=@ define is not present.
3 - Missing include of target/generic/rte.vars.mk file which includes
the information to locate the rte_config.h and other DPDK include files.

A patch like this one was submitted before and was rejected because it seemed it was not required, because target/generic/rte.vars.mk already included by rte.vars.mk.

This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk makefile fragments.

---
 mk/rte.extvars.mk | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)
mk/rte.extvars.mk +34 -2
index 3e5a990..e6c6401 100644
@@ -30,8 +30,19 @@
 #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 #
-# directory where sources are located
+# To be included at the beginning of all RTE external user Makefiles. This
+# .mk will define the RTE environment variables by including the
+# config file of SDK. It also includes the config file from external
+# application if any.
 #
+
+ifeq ($(RTE_SDK),)
+$(error RTE_SDK is not defined)
+endif
+ifeq ($(wildcard $(RTE_SDK)),)
+$(error RTE_SDK variable points to an invalid location)
+endif
+
 ifdef S
 ifeq ("$(origin S)", "command line")
 RTE_SRCDIR := $(abspath $(S))
@@ -40,6 +51,16 @@ endif
 RTE_SRCDIR  ?= $(CURDIR)
 export RTE_SRCDIR
 
+# define Q to '@' or not. $(Q) is used to prefix all shell commands to
+# be executed silently.
+Q=@
+ifdef V
+ifeq ("$(origin V)", "command line")
+Q=
+endif
+endif
+export Q
+
 #
 # Makefile to call once $(RTE_OUTPUT) is created
 #
@@ -51,6 +72,13 @@ endif
 RTE_EXTMK ?= $(RTE_SRCDIR)/Makefile
 export RTE_EXTMK
 
+# RTE_TARGET is deducted from config when we are building the SDK.
+# Else, when building an external app, RTE_TARGET must be specified
+# by the user.
+ifeq ($(RTE_TARGET),)
+$(error RTE_TARGET is not defined)
+endif
+
 RTE_SDK_BIN := $(RTE_SDK)/$(RTE_TARGET)
 
 #
@@ -78,4 +106,8 @@ RTE_MACHINE := $(CONFIG_RTE_MACHINE:"%"=%)
 RTE_EXEC_ENV := $(CONFIG_RTE_EXEC_ENV:"%"=%)
 RTE_TOOLCHAIN := $(CONFIG_RTE_TOOLCHAIN:"%"=%)
 
-
+ifneq ($(wildcard $(RTE_SDK)/mk/target/$(RTE_TARGET)/rte.vars.mk),)
+include $(RTE_SDK)/mk/target/$(RTE_TARGET)/rte.vars.mk
+else
+include $(RTE_SDK)/mk/target/generic/rte.vars.mk
+endif
Signature · 1 line
2.3.0

A message that is not in the archive (sent off-list or not yet received): 1425142563-27185-1-git-send-email-keith.wiles-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org

#2 lore raw

Hi Keith,

Quoted 12 lines · Keith Wiles

When building an external application like Pktgen and using the proper makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk files are missing some defines/includes.

1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose ouput
as the Q=@ define is not present.
3 - Missing include of target/generic/rte.vars.mk file which includes
the information to locate the rte_config.h and other DPDK include files.

Quoted 7 lines

A patch like this one was submitted before and was rejected because it seemed it was not required, because target/generic/rte.vars.mk already included by rte.vars.mk.

This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk makefile fragments.

I still not understand what is your problem. If you take an example from dpdk, let's say examples/l2fwd.

  cd test
  # compile dpdk
  git clone http://dpdk.org/git/dpdk
  cd dpdk
  DPDK=${PWD}
  make -j8 install T=x86_64-native-linuxapp-gcc
  cd ..
  # copy l2fwd in an external directory
  cp -r dpdk/examples/l2fwd .
  cd l2fwd
  # build it
  make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK}

So if you use a Makefile similar to l2fwd, I think it should work. As I explained in [1], you need to include "rte.vars.mk" at the beginning of the Makefile, not "rte.extvars.mk". But you will use "rte.extapp.mk" at the end, like in l2fwd Makefile.

Regards,
Olivier

[1] http://dpdk.org/ml/archives/dev/2015-February/013301.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw
Quoted 28 lines · Olivier MATZ

Hi Keith,

Quoted 13 lines · Keith Wiles

When building an external application like Pktgen and using the proper makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk
files are missing some defines/includes.

1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose ouput
as the Q=@ define is not present.
3 - Missing include of target/generic/rte.vars.mk file which includes
the information to locate the rte_config.h and other DPDK include files.

Quoted 8 lines

A patch like this one was submitted before and was rejected because it seemed it was not required, because target/generic/rte.vars.mk already included by rte.vars.mk.

This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk
makefile fragments.

I still not understand what is your problem. If you take an example from dpdk, let's say examples/l2fwd.

Quoted 12 lines
  cd test
  # compile dpdk
  git clone http://dpdk.org/git/dpdk
  cd dpdk
  DPDK=${PWD}
  make -j8 install T=x86_64-native-linuxapp-gcc
  cd ..
  # copy l2fwd in an external directory
  cp -r dpdk/examples/l2fwd .
  cd l2fwd
  # build it
  make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK}

Yes, this very trivial example works, but only because the makefiles are combining the two different make fragments IMO.

Then why do we have rte.extvars.mk fragment at all if it was not to be used for building outside the DPDK build directory?
Why were the rte.extXYZ.mk make fragments created at all, but to provide a clean building system outside of DPDK build?

It seem like to me we are combining two different build systems when building the examples. If rte.extvars.mk is not used then lets delete it or replace it with a single line to include rte.vars.mk.

IMO combining the two different make fragment styles is confusing and we need to remove rte.extvars.mk or replace it with my changes or replace it with a single line just to include rte.vars.mk, pick one.

Quoted 11 lines

So if you use a Makefile similar to l2fwd, I think it should work. As I explained in [1], you need to include "rte.vars.mk" at the beginning of the Makefile, not "rte.extvars.mk". But you will use "rte.extapp.mk" at the end, like in l2fwd Makefile.

Regards,
Olivier

[1] http://dpdk.org/ml/archives/dev/2015-February/013301.html

A message that is not in the archive (sent off-list or not yet received): D11C87BC.15CE3%[email protected]

#6 lore raw

Hi Keith,

Quoted 53 lines · Keith
Quoted 28 lines · Olivier MATZ

Hi Keith,

Quoted 13 lines · Keith Wiles

When building an external application like Pktgen and using the proper makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk
files are missing some defines/includes.

1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose ouput
as the Q=@ define is not present.
3 - Missing include of target/generic/rte.vars.mk file which includes
the information to locate the rte_config.h and other DPDK include files.

Quoted 8 lines

A patch like this one was submitted before and was rejected because it seemed it was not required, because target/generic/rte.vars.mk already included by rte.vars.mk.

This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk
makefile fragments.

I still not understand what is your problem. If you take an example from dpdk, let's say examples/l2fwd.

Quoted 12 lines
   cd test
   # compile dpdk
   git clone http://dpdk.org/git/dpdk
   cd dpdk
   DPDK=${PWD}
   make -j8 install T=x86_64-native-linuxapp-gcc
   cd ..
   # copy l2fwd in an external directory
   cp -r dpdk/examples/l2fwd .
   cd l2fwd
   # build it
   make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK}

Yes, this very trivial example works, but only because the makefiles are combining the two different make fragments IMO.

Then why do we have rte.extvars.mk fragment at all if it was not to be used for building outside the DPDK build directory?
Why were the rte.extXYZ.mk make fragments created at all, but to provide a clean building system outside of DPDK build?

Quoted 7 lines

It seem like to me we are combining two different build systems when building the examples. If rte.extvars.mk is not used then lets delete it or replace it with a single line to include rte.vars.mk.

IMO combining the two different make fragment styles is confusing and we need to remove rte.extvars.mk or replace it with my changes or replace it with a single line just to include rte.vars.mk, pick one.

The examples and the documentation say to use "rte.vars.mk" for external applications. It's like this since the beginning, so changing the behavior now should be done with care to avoid breaking the working applications. I don't think it's a good idea.

I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid people doing this mistake again, what do you think?

Regards,
Olivier

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw

Hi Olivier

Quoted 70 lines · Olivier MATZ

Hi Keith,

Quoted 58 lines · Keith
Quoted 32 lines · Olivier MATZ

Hi Keith,

Quoted 16 lines · Keith Wiles

When building an external application like Pktgen and using the proper makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk
files are missing some defines/includes.

1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose
ouput
as the Q=@ define is not present.
3 - Missing include of target/generic/rte.vars.mk file which
includes
the information to locate the rte_config.h and other DPDK
include
files.

Quoted 8 lines

A patch like this one was submitted before and was rejected because it seemed it was not required, because target/generic/rte.vars.mk already included by rte.vars.mk.

This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk
makefile fragments.

I still not understand what is your problem. If you take an example from
dpdk, let's say examples/l2fwd.

Quoted 12 lines
   cd test
   # compile dpdk
   git clone http://dpdk.org/git/dpdk
   cd dpdk
   DPDK=${PWD}
   make -j8 install T=x86_64-native-linuxapp-gcc
   cd ..
   # copy l2fwd in an external directory
   cp -r dpdk/examples/l2fwd .
   cd l2fwd
   # build it
   make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK}

Yes, this very trivial example works, but only because the makefiles are combining the two different make fragments IMO.

Then why do we have rte.extvars.mk fragment at all if it was not to be used for building outside the DPDK build directory?
Why were the rte.extXYZ.mk make fragments created at all, but to provide a
clean building system outside of DPDK build?

Quoted 8 lines

It seem like to me we are combining two different build systems when building the examples. If rte.extvars.mk is not used then lets delete it or replace it with a single line to include rte.vars.mk.

IMO combining the two different make fragment styles is confusing and we need to remove rte.extvars.mk or replace it with my changes or replace it
with a single line just to include rte.vars.mk, pick one.

Quoted 7 lines

The examples and the documentation say to use "rte.vars.mk" for external applications. It's like this since the beginning, so changing the behavior now should be done with care to avoid breaking the working applications. I don't think it's a good idea.

I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid people doing this mistake again, what do you think?

Instead of moving the file and someone using it anyway (as it is broke IMO) lets just replace the content of the file with a single line 'include rte.vars.mk' and we solve the problem. Plus this solves my symmetry problem :-)

Regards
++Keith

Quoted 4 lines

Regards,
Olivier

A message that is not in the archive (sent off-list or not yet received): D11C9283.15D46%[email protected]

#10 lore raw
Quoted 85 lines · Keith

Hi Olivier

Quoted 81 lines · Olivier MATZ

Hi Keith,

Quoted 67 lines · Keith
Quoted 35 lines · Olivier MATZ

Hi Keith,

Quoted 23 lines · Keith Wiles

When building an external application like Pktgen and using the proper
makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk
files are missing some defines/includes.

1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose
ouput
as the Q=@ define is not present.
3 - Missing include of target/generic/rte.vars.mk file which
includes
the information to locate the rte_config.h and other DPDK
include
files.

A patch like this one was submitted before and was rejected because it
seemed it was not required, because target/generic/rte.vars.mk already
included by rte.vars.mk.

Quoted 4 lines

This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk
makefile fragments.

I still not understand what is your problem. If you take an example from
dpdk, let's say examples/l2fwd.

Quoted 12 lines
   cd test
   # compile dpdk
   git clone http://dpdk.org/git/dpdk
   cd dpdk
   DPDK=${PWD}
   make -j8 install T=x86_64-native-linuxapp-gcc
   cd ..
   # copy l2fwd in an external directory
   cp -r dpdk/examples/l2fwd .
   cd l2fwd
   # build it
   make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK}

Yes, this very trivial example works, but only because the makefiles are
combining the two different make fragments IMO.

Then why do we have rte.extvars.mk fragment at all if it was not to be used for building outside the DPDK build directory?
Why were the rte.extXYZ.mk make fragments created at all, but to provide a
clean building system outside of DPDK build?

It seem like to me we are combining two different build systems when building the examples. If rte.extvars.mk is not used then lets delete it
or replace it with a single line to include rte.vars.mk.

Quoted 5 lines

IMO combining the two different make fragment styles is confusing and we
need to remove rte.extvars.mk or replace it with my changes or replace it
with a single line just to include rte.vars.mk, pick one.

The examples and the documentation say to use "rte.vars.mk" for external applications. It's like this since the beginning, so changing the behavior now should be done with care to avoid breaking the working applications. I don't think it's a good idea.

Quoted 7 lines
Quoted 2 lines

I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid people doing this mistake again, what do you think?

Instead of moving the file and someone using it anyway (as it is broke IMO) lets just replace the content of the file with a single line 'include rte.vars.mk' and we solve the problem. Plus this solves my symmetry problem :-)

The file can not be replaced with a single include line as it already includes rte.extvars.mk :-(

OK lets just move it to mk/internal and I will submit a patch for that change.

Quoted 8 lines

Regards
++Keith

Quoted 4 lines

Regards,
Olivier

#11 lore raw
In reply to a message not in the archive

Hi Keith,

Quoted 79 lines · Keith

Hi Olivier

Quoted 75 lines · Olivier MATZ

Hi Keith,

Quoted 58 lines · Keith
Quoted 32 lines · Olivier MATZ

Hi Keith,

Quoted 16 lines · Keith Wiles

When building an external application like Pktgen and using the proper makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk
files are missing some defines/includes.

1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose
ouput
as the Q=@ define is not present.
3 - Missing include of target/generic/rte.vars.mk file which
includes
the information to locate the rte_config.h and other DPDK
include
files.

Quoted 8 lines

A patch like this one was submitted before and was rejected because it seemed it was not required, because target/generic/rte.vars.mk already included by rte.vars.mk.

This is not the cause for external applications like Pktgen which are built outside of the RTE_SDK directory and only include the rte.extXYZ.mk
makefile fragments.

I still not understand what is your problem. If you take an example from
dpdk, let's say examples/l2fwd.

Quoted 12 lines

cd test
# compile dpdk
git clone http://dpdk.org/git/dpdk
cd dpdk
DPDK=${PWD}
make -j8 install T=x86_64-native-linuxapp-gcc
cd ..
# copy l2fwd in an external directory
cp -r dpdk/examples/l2fwd .
cd l2fwd
# build it
make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK}

Yes, this very trivial example works, but only because the makefiles are combining the two different make fragments IMO.

Then why do we have rte.extvars.mk fragment at all if it was not to be used for building outside the DPDK build directory?
Why were the rte.extXYZ.mk make fragments created at all, but to provide a
clean building system outside of DPDK build?

Quoted 8 lines

It seem like to me we are combining two different build systems when building the examples. If rte.extvars.mk is not used then lets delete it or replace it with a single line to include rte.vars.mk.

IMO combining the two different make fragment styles is confusing and we need to remove rte.extvars.mk or replace it with my changes or replace it
with a single line just to include rte.vars.mk, pick one.

The examples and the documentation say to use "rte.vars.mk" for external applications. It's like this since the beginning, so changing the behavior now should be done with care to avoid breaking the working applications. I don't think it's a good idea.

Quoted 7 lines
Quoted 2 lines

I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid people doing this mistake again, what do you think?

Instead of moving the file and someone using it anyway (as it is broke IMO) lets just replace the content of the file with a single line 'include rte.vars.mk' and we solve the problem. Plus this solves my symmetry problem :-)

I don't understand why would someone include this file directly? What is the reason you did that at the first place? Usually, people start from an example or the documentation, which are both correct.

We cannot replace the content of rte.extvars.mk by an include to rte.vars.mk because currently rte.vars.mk includes rte.extvars.mk. To me, the only problem is that rte.extvars.mk should be marked as internal.

Allowing to include rte.extvars.mk in dpdk to fix the behavior of pktgen makefiles does not seem to be a good argument. Why not fixing pktgen instead? What is broken in dpdk?

Regards,
Olivier

A message that is not in the archive (sent off-list or not yet received): [email protected]

#13 lore raw

Hi Olivier,

Quoted 104 lines · Olivier MATZ

Hi Keith,

Quoted 87 lines · Keith

Hi Olivier

Quoted 83 lines · Olivier MATZ

Hi Keith,

Quoted 68 lines · Keith
Quoted 36 lines · Olivier MATZ

Hi Keith,

Quoted 23 lines · Keith Wiles

When building an external application like Pktgen and using the proper
makefile fragments rte.extXYZ.mk NOT rte.XYZ.mk files as you would use with example applications in the same RTE_SDK directory the rte.extXYZ.mk
files are missing some defines/includes.

1 - Add missing tests for RTE_SDK/RTE_TARGET not defined code. 2 - The build of external applications are forced to be verbose
ouput
as the Q=@ define is not present.
3 - Missing include of target/generic/rte.vars.mk file which
includes
the information to locate the rte_config.h and other DPDK
include
files.

A patch like this one was submitted before and was rejected because it
seemed it was not required, because target/generic/rte.vars.mk already
included by rte.vars.mk.

Quoted 5 lines

This is not the cause for external applications like Pktgen which are
built outside of the RTE_SDK directory and only include the rte.extXYZ.mk
makefile fragments.

I still not understand what is your problem. If you take an example from
dpdk, let's say examples/l2fwd.

Quoted 12 lines

cd test
# compile dpdk
git clone http://dpdk.org/git/dpdk
cd dpdk
DPDK=${PWD}
make -j8 install T=x86_64-native-linuxapp-gcc
cd ..
# copy l2fwd in an external directory
cp -r dpdk/examples/l2fwd .
cd l2fwd
# build it
make RTE_TARGET=x86_64-native-linuxapp-gcc RTE_SDK=${DPDK}

Yes, this very trivial example works, but only because the makefiles are
combining the two different make fragments IMO.

Then why do we have rte.extvars.mk fragment at all if it was not to be used for building outside the DPDK build directory?
Why were the rte.extXYZ.mk make fragments created at all, but to provide a
clean building system outside of DPDK build?

It seem like to me we are combining two different build systems when building the examples. If rte.extvars.mk is not used then lets delete it
or replace it with a single line to include rte.vars.mk.

Quoted 5 lines

IMO combining the two different make fragment styles is confusing and we
need to remove rte.extvars.mk or replace it with my changes or replace it
with a single line just to include rte.vars.mk, pick one.

The examples and the documentation say to use "rte.vars.mk" for external
applications. It's like this since the beginning, so changing the behavior now should be done with care to avoid breaking the working applications. I don't think it's a good idea.

Quoted 8 lines
Quoted 2 lines

I would prefer to move add rte.extvars.mk in dpdk/mk/internal to avoid people doing this mistake again, what do you think?

Instead of moving the file and someone using it anyway (as it is broke IMO) lets just replace the content of the file with a single line 'include
rte.vars.mk' and we solve the problem. Plus this solves my symmetry problem :-)

I don't understand why would someone include this file directly? What is the reason you did that at the first place? Usually, people start from an example or the documentation, which are both correct.

Quoted 8 lines

We cannot replace the content of rte.extvars.mk by an include to rte.vars.mk because currently rte.vars.mk includes rte.extvars.mk. To me, the only problem is that rte.extvars.mk should be marked as internal.

Allowing to include rte.extvars.mk in dpdk to fix the behavior of pktgen makefiles does not seem to be a good argument. Why not fixing pktgen instead? What is broken in dpdk?

I just posted your point before you finished you email and will submit a patch to move rte.extvars.mk to mk/rte.extvars.mk would that work?

Quoted 4 lines

Regards,
Olivier