dpdk proposal installation process

PATCH New Arevalo, Mario Alfredo C 6 participants 13 messages Oct 20, 2015 – Nov 27, 2015

Share & tools

Arevalo, Mario Alfredo C

author

2015-10-20 00:21 UTC #0 lore raw

Hi folks,

Good day, this is a proposal in order to improve the dpdk install process,
I would like to know your point of view about the next points according to previous conversations :) in order to create a new patches version.

1) I think the first thing that I have to be aware is "compatibility", the new changes won't affect the current dpdk behaviour.

2) Create new makefile rules, these rules is going to install dpdk files in default paths, however the linux distributions don't use the same paths for their files, the linux distribution and the architecture can be factor for different path as Panu commented in previous conversations, he is right, then all variables could be overridden, the variables names for the user can be included in documentation. Also an option could be a configuration file for paths, however I'm not sure.

3) The default paths for dpdk in order to follow a hierarchy, however the variable with those values can be overridden.

-install-bin --> /usr/bin.
-install-headers --> /usr/include/dpdk
-install-lib --> /usr/lib64
-install-doc --> /usr/share/doc/dpdk
-install-mod --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
else KERNEL_DIR=/boot/modules).
-install-sdk --> /usr/share/dpdk and call install-headers ).
-install-fhs --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)

4) I'm going to take account all feedback about variables, paths etc for the new version :).

Thank you so much for your help.

Mario.

Bruce Richardson

2015-10-20 09:18 UTC #1 lore raw

Quoted 28 lines · Mario Alfredo C

Hi folks,
Good day, this is a proposal in order to improve the dpdk install process,
I would like to know your point of view about the next points according to previous conversations :) in order to create a new patches version.
1) I think the first thing that I have to be aware is "compatibility", the new changes won't affect the current dpdk behaviour.
2) Create new makefile rules, these rules is going to install dpdk files in default paths, however the linux distributions don't use the same paths for their files, the linux distribution and the architecture can be factor for different path as Panu commented in previous conversations, he is right, then all variables could be overridden, the variables names for the user can be included in documentation. Also an option could be a configuration file for paths, however I'm not sure.
3) The default paths for dpdk in order to follow a hierarchy, however the variable with those values can be overridden.
(quoted diff) +0 -7
-install-bin          --> /usr/bin.
-install-headers  --> /usr/include/dpdk
-install-lib           --> /usr/lib64
-install-doc         --> /usr/share/doc/dpdk
-install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
                                else KERNEL_DIR=/boot/modules).
-install-sdk         --> /usr/share/dpdk and call install-headers ).
-install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)

Quoted 6 lines

4) I'm going to take account all feedback about variables, paths etc for the new version :).
Thank you so much for your help.
Mario.

Hi Mario,

that seems like a lot of commands to add - are they all individually needed?

In terms of where things go, should the "usr" part not a) be configurable via a parameter, and b) default to "/usr/local" as that's where user-installed software from outside the packaging system normally gets put.

/Bruce

Olivier MATZ

2015-10-21 19:15 UTC #2 lore raw

Hi Mario,

Quoted 9 lines · Bruce Richardson

Quoted 8 lines · Mario Alfredo C
Hi folks,
Good day, this is a proposal in order to improve the dpdk install process,
I would like to know your point of view about the next points according to previous conversations :) in order to create a new patches version.
1) I think the first thing that I have to be aware is "compatibility", the new changes won't affect the current dpdk behaviour.

Yes. As I stated in a previous mail, I think nobody uses the current "make install" without specifying T= as the default value is to build and install for all targets.

My suggestion is:

- rename the previous "install" target. The name could probably be "mbuild" (for multiple builds). Other ideas are welcome.

- when "make install" is invoked with T= argument, call the mbuild target to have the same behavior than before. This compat layer could be removed in the future.

- when "make install" is invoked without T=, it installs the fhs.

Quoted 6 lines

Quoted 6 lines
2) Create new makefile rules, these rules is going to install dpdk files in default paths, however the linux distributions don't use the same paths for their files, the linux distribution and the architecture can be factor for different path as Panu commented in previous conversations, he is right, then all variables could be overridden, the variables names for the user can be included in documentation. Also an option could be a configuration file for paths, however I'm not sure.

I think having variables is ok.

Quoted 6 lines

Review
Quoted 6 lines
3) The default paths for dpdk in order to follow a hierarchy, however the variable with those values can be overridden.
(quoted diff) +0 -3
-install-bin          --> /usr/bin.
-install-headers  --> /usr/include/dpdk
-install-lib           --> /usr/lib64

I remember Panu suggested to have /usr/lib by default. I also think /usr/lib a better default value: some distributions use /usr/lib for 64 bits libs, but we never have 32 bits libs in /usr/lib64.

Quoted 3 lines

Review

Quoted 3 lines

(quoted diff) +0 -2

-install-doc         --> /usr/share/doc/dpdk
-install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
                                else KERNEL_DIR=/boot/modules).

I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?

Quoted 12 lines

Quoted 9 lines
(quoted diff) +0 -2
-install-sdk         --> /usr/share/dpdk and call install-headers ).
-install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)
4) I'm going to take account all feedback about variables, paths etc for the new version :).
Thank you so much for your help.
Mario.
Hi Mario,

Quoted 5 lines

that seems like a lot of commands to add - are they all individually needed?
In terms of where things go, should the "usr" part not a) be configurable via a parameter, and b) default to "/usr/local" as that's where user-installed software from outside the packaging system normally gets put.

A PREFIX variable would do the job.
About the default to /usr or /usr/local, I agree that /usr/local looks more usual, and I don't think it's a problem for packaging as soon as it can be overridden.

Regards,
Olivier

Panu Matilainen

2015-10-22 05:55 UTC #3 lore raw

Quoted 19 lines · Olivier MATZ

Hi Mario,
Quoted 9 lines · Bruce Richardson
Quoted 8 lines · Mario Alfredo C
Hi folks,
Good day, this is a proposal in order to improve the dpdk install process,
I would like to know your point of view about the next points according to previous conversations :) in order to create a new patches version.
1) I think the first thing that I have to be aware is "compatibility", the new changes won't affect the current dpdk behaviour.
Yes. As I stated in a previous mail, I think nobody uses the current "make install" without specifying T= as the default value is to build and install for all targets.
My suggestion is:

Quoted 8 lines

- rename the previous "install" target. The name could probably be "mbuild" (for multiple builds). Other ideas are welcome.
- when "make install" is invoked with T= argument, call the mbuild target to have the same behavior than before. This compat layer could be removed in the future.
- when "make install" is invoked without T=, it installs the fhs.

Nice, this sounds like the best of both worlds.

Quoted 13 lines

Quoted 7 lines
Quoted 7 lines
2) Create new makefile rules, these rules is going to install dpdk files in default paths, however the linux distributions don't use the same paths for their files, the linux distribution and the architecture can be factor for different path as Panu commented in previous conversations, he is right, then all variables could be overridden, the variables names for the user can be included in documentation. Also an option could be a configuration file for paths, however I'm not sure.
I think having variables is ok.
Quoted 3 lines
Quoted 3 lines
3) The default paths for dpdk in order to follow a hierarchy, however the variable with those values can be overridden.

Quoted 8 lines

Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +0 -3
-install-bin          --> /usr/bin.
-install-headers  --> /usr/include/dpdk
-install-lib           --> /usr/lib64
I remember Panu suggested to have /usr/lib by default. I also think /usr/lib a better default value: some distributions use /usr/lib for 64 bits libs, but we never have 32 bits libs in /usr/lib64.

Yes, just stick /usr/lib there and be done with it, lib64 is not a good default for these very reasons.

Quoted 20 lines

Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +0 -2
-install-doc         --> /usr/share/doc/dpdk
-install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
                                 else KERNEL_DIR=/boot/modules).
I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?
Quoted 14 lines
Quoted 9 lines
(quoted diff) +0 -2
-install-sdk         --> /usr/share/dpdk and call install-headers ).
-install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)
4) I'm going to take account all feedback about variables, paths etc for the new version :).
Thank you so much for your help.
Mario.
Hi Mario,
that seems like a lot of commands to add - are they all individually needed?

Quoted 8 lines

Quoted 3 lines
In terms of where things go, should the "usr" part not a) be configurable via a parameter, and b) default to "/usr/local" as that's where user-installed software from outside the packaging system normally gets put.
A PREFIX variable would do the job.
About the default to /usr or /usr/local, I agree that /usr/local looks more usual, and I don't think it's a problem for packaging as soon as it can be overridden.

Yeah, PREFIX support would be nice, and defaulting that to /usr/local would be the right thing.

- Panu -

Quoted 5 lines

Regards,
Olivier

Bruce Richardson

2015-10-22 14:57 UTC #4 lore raw

Quoted 85 lines · Panu Matilainen

Quoted 19 lines · Olivier MATZ
Hi Mario,
Quoted 9 lines · Bruce Richardson
Quoted 8 lines · Mario Alfredo C
Hi folks,
Good day, this is a proposal in order to improve the dpdk install process,
I would like to know your point of view about the next points according to previous conversations :) in order to create a new patches version.
1) I think the first thing that I have to be aware is "compatibility", the new changes won't affect the current dpdk behaviour.
Yes. As I stated in a previous mail, I think nobody uses the current "make install" without specifying T= as the default value is to build and install for all targets.
My suggestion is:
Quoted 8 lines
- rename the previous "install" target. The name could probably be "mbuild" (for multiple builds). Other ideas are welcome.
- when "make install" is invoked with T= argument, call the mbuild target to have the same behavior than before. This compat layer could be removed in the future.
- when "make install" is invoked without T=, it installs the fhs.
Nice, this sounds like the best of both worlds.
Quoted 13 lines
Quoted 7 lines
Quoted 7 lines
2) Create new makefile rules, these rules is going to install dpdk files in default paths, however the linux distributions don't use the same paths for their files, the linux distribution and the architecture can be factor for different path as Panu commented in previous conversations, he is right, then all variables could be overridden, the variables names for the user can be included in documentation. Also an option could be a configuration file for paths, however I'm not sure.
I think having variables is ok.
Quoted 3 lines
Quoted 3 lines
3) The default paths for dpdk in order to follow a hierarchy, however the variable with those values can be overridden.
Quoted 8 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +0 -3
-install-bin          --> /usr/bin.
-install-headers  --> /usr/include/dpdk
-install-lib           --> /usr/lib64
I remember Panu suggested to have /usr/lib by default. I also think /usr/lib a better default value: some distributions use /usr/lib for 64 bits libs, but we never have 32 bits libs in /usr/lib64.
Yes, just stick /usr/lib there and be done with it, lib64 is not a good default for these very reasons.
Quoted 20 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +0 -2
-install-doc         --> /usr/share/doc/dpdk
-install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
                                else KERNEL_DIR=/boot/modules).
I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?
Quoted 14 lines
Quoted 9 lines
(quoted diff) +0 -2
-install-sdk         --> /usr/share/dpdk and call install-headers ).
-install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)
4) I'm going to take account all feedback about variables, paths etc for the new version :).
Thank you so much for your help.
Mario.
Hi Mario,
that seems like a lot of commands to add - are they all individually needed?
Quoted 8 lines
Quoted 3 lines
In terms of where things go, should the "usr" part not a) be configurable via a parameter, and b) default to "/usr/local" as that's where user-installed software from outside the packaging system normally gets put.
A PREFIX variable would do the job.
About the default to /usr or /usr/local, I agree that /usr/local looks more usual, and I don't think it's a problem for packaging as soon as it can be overridden.

Quoted 11 lines

Yeah, PREFIX support would be nice, and defaulting that to /usr/local would be the right thing.
- Panu -
Quoted 5 lines
Regards,
Olivier

Can I throw a completely different suggestion into the mix?

Can we make use of the fact that make config creates a directory called "build" by default. Then running "make" alone in that directory does the expected behaviour of a compile of the whole sdk. How about having "make install" in the build directory behave like a generic "make install" call for other packages?

I'm imagining the following sequence of steps to install:

	./configure --machine=[default|native|other] 
		# configure is a simple script that just calls "make config T=..."
	cd build
	make
	make install

Thoughts?

/Bruce

Arevalo, Mario Alfredo C

author

2015-10-26 16:18 UTC #5 lore raw

Quoted 134 lines

-----Original Message-----
From: Richardson, Bruce
Sent: Thursday, October 22, 2015 9:57 AM
To: Panu Matilainen
Cc: Olivier MATZ; Arevalo, Mario Alfredo C; [email protected]
Subject: Re: [dpdk-dev] dpdk proposal installation process
Quoted 5 lines · Panu Matilainen
Quoted 4 lines · Olivier MATZ
Hi Mario,
Quoted 1 line · Bruce Richardson
On Tue, Oct 20, 2015 at 12:21:00AM +0000, Arevalo, Mario Alfredo C
Quoted 5 lines · wrote:
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Hi folks,
Good day, this is a proposal in order to improve the dpdk
install process, I would like to know your point of view about the next points according to previous conversations :) in order to create a
new patches version.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
1) I think the first thing that I have to be aware is "compatibility", the new changes won't affect the current dpdk
behaviour.
Quoted 10 lines
Quoted 10 lines
Yes. As I stated in a previous mail, I think nobody uses the current "make install" without specifying T= as the default value is to build and install for all targets.
My suggestion is:
- rename the previous "install" target. The name could probably be "mbuild" (for multiple builds). Other ideas are welcome.
Quoted 14 lines
Quoted 5 lines
- when "make install" is invoked with T= argument, call the mbuild target to have the same behavior than before. This compat layer could be removed in the future.
- when "make install" is invoked without T=, it installs the fhs.
Nice, this sounds like the best of both worlds.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
2) Create new makefile rules, these rules is going to install dpdk files in default paths, however the linux distributions don't use the same paths for their files, the linux distribution and the architecture can be factor for different path as Panu commented in previous conversations, he is right, then all variables could be
overridden, the variables names for the user can be included in documentation.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Also an option could be a configuration file for paths, however I'm not
sure.
Quoted 15 lines
Quoted 6 lines
I think having variables is ok.
Quoted 3 lines
Quoted 3 lines
3) The default paths for dpdk in order to follow a hierarchy, however the variable with those values can be overridden.
Quoted 8 lines
Quoted 3 lines
Review
Quoted 3 lines
(quoted diff) +0 -3
-install-bin          --> /usr/bin.
-install-headers  --> /usr/include/dpdk
-install-lib           --> /usr/lib64
I remember Panu suggested to have /usr/lib by default.
I also think /usr/lib a better default value: some distributions use /usr/lib for 64 bits libs, but we never have 32 bits libs in /usr/lib64.
Quoted 5 lines
Yes, just stick /usr/lib there and be done with it, lib64 is not a good default for these very reasons.
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +0 -2
-install-doc         --> /usr/share/doc/dpdk
-install-mod        --> if RTE_EXEC_ENV=linuxapp then
KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
else KERNEL_DIR=/boot/modules).
I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +0 -2
-install-sdk         --> /usr/share/dpdk and call install-headers ).
-install-fhs          --> call install-libraries, install-mod, install-bin and install-
doc (maybe install-headers)
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
4) I'm going to take account all feedback about variables, paths etc for
the new version :).
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
Thank you so much for your help.
Mario.
Hi Mario,
that seems like a lot of commands to add - are they all individually
needed?
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
In terms of where things go, should the "usr" part not a) be configurable via a parameter, and b) default to "/usr/local" as that's where user-installed software from outside the packaging system
normally gets put.
Quoted 6 lines
Quoted 5 lines
A PREFIX variable would do the job.
About the default to /usr or /usr/local, I agree that /usr/local looks more usual, and I don't think it's a problem for packaging as soon as it can be overridden.
Quoted 11 lines
Yeah, PREFIX support would be nice, and defaulting that to /usr/local would be the right thing.
- Panu -
Quoted 5 lines
Regards,
Olivier
Can I throw a completely different suggestion into the mix?
Can we make use of the fact that make config creates a directory called "build"
by default. Then running "make" alone in that directory does the expected behaviour of a compile of the whole sdk. How about having "make install" in the build directory behave like a generic "make install" call for other packages?
I'm imagining the following sequence of steps to install:
	./configure --machine=[default|native|other]
		# configure is a simple script that just calls "make config T=..."
	cd build
	make
	make install

Quoted 3 lines

Thoughts?
/Bruce

Hi Guys,

Thank you so much for your feedback, about your last comments, I understood the next:

1)
-"make install" using "T" variable:
This command will use a rule for the current behaviour called "mbuild"

-"make install" without "T" variable:
This command will install everything: headers, libraries, modules, apps, docs, and sdk files (config files, make files, scripts and examples).

2) About variables, you rigth, KMOD_DIR sounds better than KERNEL_DIR, and LIB_DIR should be /usr/lib by default.

3)
-To use a "prefix variable"
-This prefix variable will be "/usr/local" by default. -This prefix variable can be overridden.

Example:
$(DESTDIR)/$(PREFIX)/$(INCLUDE_DIR)

4) What do you think about the Bruce's proposal?

Bruce Richardson

2015-10-27 14:25 UTC #6 lore raw

compile and install using configure-make-make_install

Just clarify a bit further the idea I had in mind, I've put together the following two example patches. After applying these patches the following sequence of operations works to install dpdk libraries and headers into /opt (/opt/lib/dpdk and /opt/include/dpdk respectively). Also the machine type is changed from native to default, and the settings for KNI are disabled.

	./configure --prefix=/opt --machine=default \
		--config=RTE_LIBRTE_KNI=n \
		--config=CONFIG_RTE_KNI_KMOD=n
	cd build
	make 
	sudo make install

NOTE: I'm not suggesting this is ready for inclusion in DPDK in any shape or form, I'm just trying to make clear what my suggestion was referring to, and to trigger more discussion on the topic.

Bruce Richardson (2):
gen-build-mk: add "make install" option to build dir add example configure script

 configure               | 59 +++++++++++++++++++++++++++++++++++++++++++++++++
 scripts/gen-build-mk.sh | 13 +++++++++++
 2 files changed, 72 insertions(+)
 create mode 100755 configure

Signature · 1 line

2.4.3

Bruce Richardson

2015-10-27 14:25 UTC #7 lore raw

PATCH 1/2 gen-build-mk: add "make install" option to build dir

Add in a "make install" command to the makefile for a DPDK build directory. Once inside the directory, make and make install will behave as normal for opensource packages, compiling and installing in the filesystem the DPDK code.

Signed-off-by Bruce Richardson <[email protected]>

---
 scripts/gen-build-mk.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)

scripts/gen-build-mk.sh +13 -0

index 5c12813..2387cf6 100755
@@ -36,6 +36,9 @@
 #   $1: path of project src root
 #   $2: path of build dir (can be relative to $1)
 
+if [ -z "$RTE_PREFIX" ] ; then
+	RTE_PREFIX=/usr/local
+fi
 echo "# Automatically generated by gen-build-mk.sh"
 echo
 echo "ifdef O"

@@ -46,8 +49,18 @@ echo "endif"
 echo
 echo "MAKEFLAGS += --no-print-directory"
 echo
+echo "INSTALL = install -v"
+echo
 echo "all:"
 echo "	@\$(MAKE) -C $1 O=$2"
 echo
+echo "install:"
+echo "	@\$(INSTALL) -d $RTE_PREFIX/lib/dpdk $RTE_PREFIX/include/dpdk"
+echo "	@\$(INSTALL) -d $RTE_PREFIX/include/dpdk/exec-env $RTE_PREFIX/include/dpdk/generic"
+echo "	@\$(INSTALL) lib/* $RTE_PREFIX/lib/dpdk"
+echo "	@\$(INSTALL) include/*.h $RTE_PREFIX/include/dpdk"
+echo "	@\$(INSTALL) include/exec-env/*.h $RTE_PREFIX/include/dpdk/exec-env"
+echo "	@\$(INSTALL) include/generic/*.h $RTE_PREFIX/include/dpdk/generic"
+echo
 echo "%::"
 echo "	@\$(MAKE) -C $1 O=$2 \$@"

Signature · 1 line

2.4.3

Bruce Richardson

2015-10-27 14:25 UTC #8 lore raw

PATCH 2/2 add example configure script

In reply to Bruce Richardson

To demonstrate the idea of using a configure-make-make install type operation for installing DPDK, add in a simple configure script in python which runs "make config T=<target>" to do most of the work and then tweaks the .config file afterwards.

Signed-off-by Bruce Richardson <[email protected]>

---
 configure | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100755 configure

configure new file +59 -0

new file mode 100755
index 0000000..e820bdd
@@ -0,0 +1,59 @@
+#! /usr/bin/env python
+
+#   BSD LICENSE
+#
+#   Copyright(c) 2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+config_append=""
+
+from getopt import getopt
+import sys, os
+
+optlist, args = getopt(sys.argv[1:], '', ['prefix=','machine=','config='])
+
+for option, arg in optlist:
+	if option == "--prefix":
+		os.environ['RTE_PREFIX'] = arg
+	elif option == "--machine":
+		config_append += "CONFIG_RTE_MACHINE=\"%s\"\n" % arg
+	elif option == "--config":
+		if not arg.startswith("CONFIG_"):
+			config_append += "CONFIG_%s\n" % arg
+		else:
+			config_append += "%s\n" % arg
+
+if "RTE_TARGET" in os.environ:
+	os.system("make config T=$RTE_TARGET")
+else:
+	os.system("make config T=x86_64-native-linuxapp-gcc")
+f = open("build/.config", "a")
+f.write(config_append)
+f.close()
+

Signature · 1 line

2.4.3

Panu Matilainen

2015-11-03 07:35 UTC #9 lore raw

Re: compile and install using configure-make-make_install

In reply to Bruce Richardson

Quoted 6 lines · Bruce Richardson

Just clarify a bit further the idea I had in mind, I've put together the following two example patches. After applying these patches the following sequence of operations works to install dpdk libraries and headers into /opt (/opt/lib/dpdk and /opt/include/dpdk respectively). Also the machine type is changed from native to default, and the settings for KNI are disabled.

Quoted 6 lines

	./configure --prefix=/opt --machine=default \
		--config=RTE_LIBRTE_KNI=n \
		--config=CONFIG_RTE_KNI_KMOD=n
	cd build
	make
	sudo make install

I think I like the idea of a configure script that hides (some of) the DPDK config peculiarities behind a more familiar looking interface. For example it could be used for figuring out the machine-os-compiler triplet automatically for the common cases.

However having "make install" behave differently depending on which directory its invoked from I dunno. Its unlike anything else out there (that I know of) and I thought the idea was to make DPDK behave more like a normal OSS citizen :)

- Panu -

Bruce Richardson

2015-11-03 10:16 UTC #10 lore raw

Re: compile and install using configure-make-make_install

Quoted 19 lines · Panu Matilainen

Quoted 6 lines · Bruce Richardson
Just clarify a bit further the idea I had in mind, I've put together the following two example patches. After applying these patches the following sequence of operations works to install dpdk libraries and headers into /opt (/opt/lib/dpdk and /opt/include/dpdk respectively). Also the machine type is changed from native to default, and the settings for KNI are disabled.
Quoted 6 lines
	./configure --prefix=/opt --machine=default \
		--config=RTE_LIBRTE_KNI=n \
		--config=CONFIG_RTE_KNI_KMOD=n
	cd build
	make
	sudo make install
I think I like the idea of a configure script that hides (some of) the DPDK config peculiarities behind a more familiar looking interface. For example it could be used for figuring out the machine-os-compiler triplet automatically for the common cases.

Quoted 7 lines

However having "make install" behave differently depending on which directory its invoked from I dunno. Its unlike anything else out there (that I know of) and I thought the idea was to make DPDK behave more like a normal OSS citizen :)
- Panu -

Yes, I would agree with your complaint.

Unfortunately, I can't see any quick-fix way for it in the short term. Ideally, I think we should look to rename the existing "make install " option to something else, and then later look to add in a proper make install like above.

For example: I would suggest in one release e.g. 2.2 to rename make install to e.g. "make sdk" or "make target" [I like make target, since it implies that you specify what the target should be with T="..."].
At that point, we add a new "make install" rule which simply prints out "make install is deprecated, use 'make target' instead."

Then we wait a couple of releases to allow everyone to switch over to the new rule. Once done, then we can look to introduce a new, more standard, "make install" type target. [And in this case, I think limiting it to a build subdir or some such would also be helpful, just to help those doing a multi-version upgrade jump]

/Bruce

Thomas Monjalon

2015-11-27 11:52 UTC #11 lore raw

In reply to Arevalo, Mario Alfredo C

2015-10-26 16:18, Arevalo, Mario Alfredo C:

Quoted 8 lines

Hi Guys,
Thank you so much for your feedback, about your last comments, I understood the next:
1)
-"make install" using "T" variable:
This command will use a rule for the current behaviour called "mbuild"

I think the ability of building several targets in one command make the build system more complex than it should be.
A target should be defined by its configured options, not a default template config.
So I suggest to drop it.

Quoted 12 lines

-"make install" without "T" variable:
This command will install everything: headers, libraries, modules, apps, docs, and sdk files (config files, make files, scripts and examples).
2) About variables, you rigth, KMOD_DIR sounds better than KERNEL_DIR, and LIB_DIR should be /usr/lib by default.
3)
(quoted diff) +0 -3
-To use a "prefix variable"
-This prefix variable will be "/usr/local" by default.
-This prefix variable can be overridden.

Quoted 2 lines

Example:
$(DESTDIR)/$(PREFIX)/$(INCLUDE_DIR)

It is a good step.
In order to be more standard, we should try to comply with these guidelines:
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
https://www.gnu.org/prep/standards/html_node/DESTDIR.html

Quoted 1 line

4) What do you think about the Bruce's proposal?

I love the idea of having a configure script.
I would add some checks to solve the dependencies (external and internal).

Marc

2015-11-27 13:16 UTC #12 lore raw

In reply to Bruce Richardson

Quoted 126 lines · Bruce Richardson

Quoted 5 lines · Panu Matilainen
Quoted 4 lines · Olivier MATZ
Hi Mario,
Quoted 1 line · Bruce Richardson
On Tue, Oct 20, 2015 at 12:21:00AM +0000, Arevalo, Mario Alfredo C
Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Hi folks,
Good day, this is a proposal in order to improve the dpdk
install process,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
I would like to know your point of view about the next points
according to
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
previous conversations :) in order to create a new patches version.
1) I think the first thing that I have to be aware is
"compatibility", the
Quoted 11 lines
Quoted 11 lines
Quoted 1 line
Quoted 1 line
new changes won't affect the current dpdk behaviour.
Yes. As I stated in a previous mail, I think nobody uses the current "make install" without specifying T= as the default value is to build and install for all targets.
My suggestion is:
- rename the previous "install" target. The name could probably be "mbuild" (for multiple builds). Other ideas are welcome.
Quoted 10 lines
Quoted 5 lines
- when "make install" is invoked with T= argument, call the mbuild target to have the same behavior than before. This compat layer could be removed in the future.
- when "make install" is invoked without T=, it installs the fhs.
Nice, this sounds like the best of both worlds.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
2) Create new makefile rules, these rules is going to install dpdk
files in
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
default paths, however the linux distributions don't use the same
paths for their
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
files, the linux distribution and the architecture can be factor for
different
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
path as Panu commented in previous conversations, he is right, then
all variables
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
could be overridden, the variables names for the user can be included
in documentation.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Also an option could be a configuration file for paths, however I'm
not sure.
Quoted 4 lines
Quoted 4 lines
I think having variables is ok.
Quoted 1 line
Quoted 1 line
3) The default paths for dpdk in order to follow a hierarchy, however
the variable
Quoted 11 lines
Quoted 10 lines
Quoted 5 lines
Review
Quoted 5 lines
with those values can be overridden.
(quoted diff) +0 -3
-install-bin          --> /usr/bin.
-install-headers  --> /usr/include/dpdk
-install-lib           --> /usr/lib64
I remember Panu suggested to have /usr/lib by default. I also think /usr/lib a better default value: some distributions use /usr/lib for 64 bits libs, but we never have 32 bits libs in /usr/lib64.
Quoted 5 lines
Yes, just stick /usr/lib there and be done with it, lib64 is not a good default for these very reasons.
Quoted 2 lines
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +0 -2
-install-doc         --> /usr/share/doc/dpdk
-install-mod        --> if RTE_EXEC_ENV=linuxapp then
KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
else KERNEL_DIR=/boot/modules).
I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?
Quoted 2 lines
Review
Quoted 2 lines
(quoted diff) +0 -2
-install-sdk         --> /usr/share/dpdk and call install-headers ).
-install-fhs          --> call install-libraries, install-mod,
install-bin and install-doc (maybe install-headers)
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
4) I'm going to take account all feedback about variables, paths etc
for the new version :).
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
Thank you so much for your help.
Mario.
Hi Mario,
that seems like a lot of commands to add - are they all individually
needed?
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
In terms of where things go, should the "usr" part not a) be
configurable via
Quoted 1 line
Quoted 1 line
Quoted 1 line
a parameter, and b) default to "/usr/local" as that's where
user-installed
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
software from outside the packaging system normally gets put.
A PREFIX variable would do the job.
About the default to /usr or /usr/local, I agree that /usr/local looks more usual, and I don't think it's a problem for packaging as soon as it can be overridden.
Yeah, PREFIX support would be nice, and defaulting that to /usr/local
would
Quoted 10 lines
be the right thing.
- Panu -
Quoted 5 lines
Regards,
Olivier
Can I throw a completely different suggestion into the mix?
Can we make use of the fact that make config creates a directory called "build"
by default. Then running "make" alone in that directory does the expected behaviour of a compile of the whole sdk. How about having "make install" in the
build directory behave like a generic "make install" call for other packages?

Quoted 7 lines

I'm imagining the following sequence of steps to install:
./configure --machine=[default|native|other]
# configure is a simple script that just calls "make
config T=..."
cd build

Why not the inverse, configure in the folder where you build so that you have all the compilation environment in the target folder (as in autoconf+automake and as of now in DPDK). You can have easily parallel builds in different folders.

Quoted 3 lines

make
make install

If you want this workflow, why not directly using autoconf + maybe the config file there is now (since there are a ton of parameters)? Putting general configuration parameters into configure.ac and leave the rest to the config files.

The PREFIX and installation of files is something that automake+autoconf solves too (probably without libtool for DPDK).

In any case, for install-fhs, I would name it install-all, to make it consistent with typical autotools build envs, which is what users are used to.

Marc

Quoted 4 lines

Thoughts?
/Bruce