Automatic device driver back-porting with media_build

DISCUSSION Mason 2 participants 25 messages Dec 17, 2015 – Dec 28, 2015 Media (V4L/DVB)

Share & tools

Mason

author

2015-12-17 12:13 UTC #0 lore raw

Hello everyone,

I have a TechnoTrend TT-TVStick CT2-4400v2 USB tuner, as described here:
http://linuxtv.org/wiki/index.php/TechnoTrend_TT-TVStick_CT2-4400

According to the article, the device is supported since kernel 3.19 and indeed, if I use a 4.1 kernel, I can pick CONFIG_DVB_USB_DVBSKY and everything seems to work.

Unfortunately (for me), I've been asked to make this driver work on an ancient 3.4 kernel.

The linuxtv article mentions:

"Drivers are included in kernel 3.17 (for version 1) and 3.19 (for version 2). They can be built with media_build for older kernels." ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This seems to imply that I can use the media_build framework to automatically (??) back-port a 3.19 driver to a 3.4 kernel? This sounds too good to be true...
How far back can I go?

http://linuxtv.org/wiki/index.php/How_to_Obtain,_Build_and_Install_V4L-DVB_Device_Drivers

I find the instructions not very clear.

I have cloned media_tree and media_build. And I have my 3.4 kernel source in a separate "my-linux-3.4" dir.

How am I supposed to tell media_build: "hey, the latest drivers are in this "media_tree" dir, I'd like you to compile this one driver for the kernel in this "my-linux-3.4" dir" ?

Note that media_build/linux has scripts which reference include/uapi which did not exist yet in 3.4

Anyway, my confusion level is at 11. I'd be very grateful if anyone here can clear some of it!

Regards.

Mauro Carvalho Chehab

2015-12-17 12:55 UTC #1 lore raw

Em Thu, 17 Dec 2015 13:13:36 +0100
Mason <[email protected]> escreveu:

Quoted 12 lines

Hello everyone,
I have a TechnoTrend TT-TVStick CT2-4400v2 USB tuner, as described here:
http://linuxtv.org/wiki/index.php/TechnoTrend_TT-TVStick_CT2-4400
According to the article, the device is supported since kernel 3.19 and indeed, if I use a 4.1 kernel, I can pick CONFIG_DVB_USB_DVBSKY and everything seems to work.
Unfortunately (for me), I've been asked to make this driver work on an ancient 3.4 kernel.

Quoted 8 lines

The linuxtv article mentions:
"Drivers are included in kernel 3.17 (for version 1) and 3.19 (for version 2). They can be built with media_build for older kernels." ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This seems to imply that I can use the media_build framework to automatically (??) back-port a 3.19 driver to a 3.4 kernel?

"automatically" is a complex word ;)

Quoted 2 lines

This sounds too good to be true...
How far back can I go?

The goal is to allow compilation since 2.6.32, but please notice that not all drivers will go that far. Basically, when the backport seems too complex, we just remove the driver from the list of drivers that are compiled for a given legacy version.

Se the file v4l/versions.txt to double-check if the drivers you need have such restrictions. I suspect that, in the specific case of DVB_USB_DVBSKY, it should compile.

That doesn't mean that it was tested there. We don't test those backports to check against regressions. We only work, at best effort basis, to make them to build. So, use it with your own risk. If you find any problems, feel free to send us patches fixing it.

Quoted 4 lines

http://linuxtv.org/wiki/index.php/How_to_Obtain,_Build_and_Install_V4L-DVB_Device_Drivers
I find the instructions not very clear.

Feel free to improve them ;)

Quoted 7 lines

I have cloned media_tree and media_build. And I have my 3.4 kernel source in a separate "my-linux-3.4" dir.
How am I supposed to tell media_build: "hey, the latest drivers are in this "media_tree" dir, I'd like you to compile this one driver for the kernel in this "my-linux-3.4" dir" ?

if the headers for version 3.4 aren't installed, see:

$ make help
...
release		- Allows changing kernel version.
		  Typical usage is:
			make release VER=2.6.12-18mdk
			(to force compiling to 2.6.12-18mdk)
			(This will work only if
			 /lib/modules/2.6.12-18mdk/build/
			 points to that kernel version)
		  Or
			make release DIR=~/linux-git
			(to force using kernel at a specific dir)
		  To use current kernel version instead:
			make release

Quoted 8 lines

Note that media_build/linux has scripts which reference include/uapi which did not exist yet in 3.4
Anyway, my confusion level is at 11. I'd be very grateful if anyone here can clear some of it!
Regards.

Mason

author

2015-12-17 13:52 UTC #2 lore raw

Hello Mauro,

Quoted 14 lines · Mauro Carvalho Chehab

Quoted 12 lines · Mason
I have a TechnoTrend TT-TVStick CT2-4400v2 USB tuner, as described here:
http://linuxtv.org/wiki/index.php/TechnoTrend_TT-TVStick_CT2-4400
According to the article, the device is supported since kernel 3.19 and indeed, if I use a 4.1 kernel, I can pick CONFIG_DVB_USB_DVBSKY and everything seems to work.
Unfortunately (for me), I've been asked to make this driver work on an ancient 3.4 kernel.
The linuxtv article mentions:

Quoted 8 lines

Quoted 6 lines
"Drivers are included in kernel 3.17 (for version 1) and 3.19 (for version 2). They can be built with media_build for older kernels." ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This seems to imply that I can use the media_build framework to automatically (??) back-port a 3.19 driver to a 3.4 kernel?
"automatically" is a complex word ;)

If I get it working, I think you can even say "auto-magically" ;-)

Quoted 11 lines

Quoted 2 lines
This sounds too good to be true...
How far back can I go?
The goal is to allow compilation since 2.6.32, but please notice that not all drivers will go that far. Basically, when the backport seems too complex, we just remove the driver from the list of drivers that are compiled for a given legacy version.
Se the file v4l/versions.txt to double-check if the drivers you need have such restrictions. I suspect that, in the specific case of DVB_USB_DVBSKY, it should compile.

That is great news.

Quoted 5 lines

That doesn't mean that it was tested there. We don't test those backports to check against regressions. We only work, at best effort basis, to make them to build. So, use it with your own risk. If you find any problems, feel free to send us patches fixing it.

My first problem is that compilation fails on the first file ;-) See attached log.

My steps are:

cd media_build/linux
make tar DIR=/tmp/sandbox/media_tree
make untar
cd ..
make release DIR=/tmp/sandbox/custom-linux-3.4
make

I will investigate and report back.

Regards.

/tmp/sandbox$ cd media_build/linux
/tmp/sandbox/media_build/linux$ make tar DIR=/tmp/sandbox/media_tree
rm -f /tmp/sandbox/media_build/linux/linux-media.tar.bz2
tar cf /tmp/sandbox/media_build/linux/linux-media.tar -C /tmp/sandbox/media_tree sound/pci/bt87x.c mm/frame_vector.c include/linux/mmc/sdio_ids.h include/sound/aci.h include/uapi/linux/usb/video.h include/linux/via-core.h include/linux/ti_wilink_st.h include/linux/dma-buf.h include/linux/fence.h include/linux/of_graph.h include/linux/kconfig.h include/linux/hdmi.h include/linux/compiler-gcc.h include/linux/dma/xilinx_dma.h include/trace/events/v4l2.h include/trace/events/vb2.h include/linux/pci_ids.h include/misc/altera.h include/uapi/linux/lirc.h include/uapi/linux/videodev2.h include/uapi/linux/meye.h include/uapi/linux/ivtv.h include/uapi/linux/ivtvfb.h include/uapi/linux/media.h include/uapi/linux/media-bus-format.h include/uapi/linux/v4l2-dv-timings.h include/uapi/linux/v4l2-controls.h include/uapi/linux/uvcvideo.h include/uapi/linux/vsp1.h include/uapi/linux/xilinx-v4l2-controls.h include/uapi/linux/smiapp.h include/uapi/linux/v4l2-subdev.h include/uapi/linux/v4l2-common.h include/uapi/linux/v4l2-mediabus.h include/linux/fixp-arith.h firmware/av7110/bootcode.bin.ihex firmware/av7110/Boot.S firmware/cpia2/stv0672_vp4.bin.ihex firmware/ihex2fw.c firmware/vicam/firmware.H16 firmware/ttusb-budget/dspbootcode.bin.ihex git --git-dir /tmp/sandbox/media_tree/.git log --pretty=oneline -n3 |sed -r 's,([\x22]),,g; s,([\x25\x5c]),\1\1,g' >git_log
perl -e 'while (<>) { $a=$1 if (m/^\s*VERSION\s*=\s*(\d+)/); $b=$1 if (m/^\s*PATCHLEVEL\s*=\s*(\d+)/); $c=$1 if (m/^\s*SUBLEVEL\s*=\s*(\d+)/); } printf "#define V4L2_VERSION %d\n", ((($a) << 16) + (($b) << 8) + ($c))' /tmp/sandbox/media_tree/Makefile > kernel_version.h
tar rvf /tmp/sandbox/media_build/linux/linux-media.tar git_log kernel_version.h
git_log
kernel_version.h

for i in drivers/media/ drivers/staging/media/ drivers/misc/altera-stapl/ include/media/ include/dt-bindings/media/ include/linux/platform_data/media/ include/uapi/linux/dvb/; do \
		if [ "`echo $i|grep Documentation`" = "" ]; then \
			dir="`(cd /tmp/sandbox/media_tree; find $i -type f -name '*.[ch]')`"; \
			dir="$dir `(cd /tmp/sandbox/media_tree; find $i -type f -name Makefile)`"; \
			dir="$dir `(cd /tmp/sandbox/media_tree; find $i -type f -name Kconfig)`"; \
			tar rvf /tmp/sandbox/media_build/linux/linux-media.tar -C /tmp/sandbox/media_tree $dir; \
		else \
			tar rvf /tmp/sandbox/media_build/linux/linux-media.tar -C /tmp/sandbox/media_tree $i; \
		fi; done; bzip2 /tmp/sandbox/media_build/linux/linux-media.tar

[snip list of 2247 files]
/tmp/sandbox/media_build/linux$ make untar
tar xfj linux-media.tar.bz2
rm -f .patches_applied .linked_dir .git_log.md5
/tmp/sandbox/media_build/linux$ cd ..
/tmp/sandbox/media_build$ make release DIR=/tmp/sandbox/custom-linux-3.4
make -C /tmp/sandbox/media_build/v4l release
make[1]: Entering directory `/tmp/sandbox/media_build/v4l'
Searching in /tmp/sandbox/custom-linux-3.4/Makefile for kernel version.
Forcing compiling to version 3.4.3913
make[1]: Leaving directory `/tmp/sandbox/media_build/v4l' /tmp/sandbox/media_build$ make
make -C /tmp/sandbox/media_build/v4l
make[1]: Entering directory `/tmp/sandbox/media_build/v4l' scripts/make_makefile.pl
Updating/Creating .config
make[2]: Entering directory `/tmp/sandbox/media_build/linux'
Applying patches for kernel 3.4.3913
patch -s -f -N -p1 -i ../backports/api_version.patch
patch -s -f -N -p1 -i ../backports/pr_fmt.patch
patch -s -f -N -p1 -i ../backports/debug.patch
patch -s -f -N -p1 -i ../backports/drx39xxj.patch
patch -s -f -N -p1 -i ../backports/v4.1_pat_enabled.patch
patch -s -f -N -p1 -i ../backports/v4.0_dma_buf_export.patch
patch -s -f -N -p1 -i ../backports/v4.0_drop_trace.patch
patch -s -f -N -p1 -i ../backports/v4.0_fwnode.patch
patch -s -f -N -p1 -i ../backports/v3.19_get_user_pages_locked.patch
Patched drivers/media/dvb-core/dvbdev.c
Patched drivers/media/v4l2-core/v4l2-dev.c
Patched drivers/media/rc/rc-main.c
make[2]: Leaving directory `/tmp/sandbox/media_build/linux'
Preparing to compile for kernel version 3.4.3913
WARNING: This is the V4L/DVB backport tree, with experimental drivers
backported to run on legacy kernels from the development tree at:
http://git.linuxtv.org/media-tree.git.
It is generally safe to use it for testing a new driver or
feature, but its usage on production environments is risky.
Don't use it in production. You've been warned.
V4L2_FLASH_LED_CLASS: Requires at least kernel 4.2.0
VIDEOBUF2_DMA_CONTIG: Requires at least kernel 3.6.0
IR_HIX5HD2: Requires at least kernel 3.10.0
IR_IMG: Requires at least kernel 3.9.0
RC_ST: Requires at least kernel 3.15.0
DVB_USB_RTL28XXU: Requires at least kernel 3.7.0
VIDEO_FB_IVTV: Requires at least kernel 3.11.0
DVB_PT3: Requires at least kernel 3.11.0
DVB_NETUP_UNIDVB: Requires at least kernel 3.7.0
VIDEO_RCAR_VIN: Requires at least kernel 3.9.0
VIDEO_XILINX: Requires at least kernel 3.17.0
VIDEO_CODA: Requires at least kernel 3.5.0
VIDEO_SH_VEU: Requires at least kernel 3.9.0
VIDEO_RENESAS_VSP1: Requires at least kernel 3.9.0
RADIO_SI4713: Requires at least kernel 3.13.0
I2C_SI4713: Requires at least kernel 3.17.0
VIDEO_ADV7183: Requires at least kernel 3.5.0
VIDEO_ADV7604: Requires at least kernel 3.17.0
VIDEO_TC358743: Requires at least kernel 3.17.0
VIDEO_OV2659: Requires at least kernel 3.5.0
VIDEO_OV9650: Requires at least kernel 3.5.0
VIDEO_VS6624: Requires at least kernel 3.5.0
VIDEO_MT9P031: Requires at least kernel 3.17.0
VIDEO_MT9T001: Requires at least kernel 3.5.0
VIDEO_MT9V032: Requires at least kernel 3.19.0
VIDEO_NOON010PC30: Requires at least kernel 3.5.0
VIDEO_M5MOLS: Requires at least kernel 3.6.0
VIDEO_S5K6AA: Requires at least kernel 3.5.0
VIDEO_S5K6A3: Requires at least kernel 3.5.0
VIDEO_S5K5BAF: Requires at least kernel 3.5.0
VIDEO_SMIAPP: Requires at least kernel 4.0.0
VIDEO_S5C73M3: Requires at least kernel 3.6.0
VIDEO_ADP1653: Requires at least kernel 3.17.0
SOC_CAMERA_OV2640: Requires at least kernel 3.17.0
MEDIA_TUNER_E4000: Requires at least kernel 3.5.0
DVB_M88DS3103: Requires at least kernel 3.8.0
DVB_TS2020: Requires at least kernel 3.8.0
DVB_RTL2830: Requires at least kernel 3.8.0
DVB_RTL2832: Requires at least kernel 3.8.0
Created default (all yes) .config file
./scripts/make_myconfig.pl
make[1]: Leaving directory `/tmp/sandbox/media_build/v4l'
make[1]: Entering directory `/tmp/sandbox/media_build/v4l' scripts/make_makefile.pl
make[1]: Leaving directory `/tmp/sandbox/media_build/v4l'
make[1]: Entering directory `/tmp/sandbox/media_build/v4l'
perl scripts/make_config_compat.pl /tmp/sandbox/custom-linux-3.4 ./.myconfig ./config-compat.h creating symbolic links...
make -C firmware prep
make[2]: Entering directory `/tmp/sandbox/media_build/v4l/firmware'
make[2]: Leaving directory `/tmp/sandbox/media_build/v4l/firmware'
make -C firmware
make[2]: Entering directory `/tmp/sandbox/media_build/v4l/firmware'
CC ihex2fw
Generating vicam/firmware.fw
Generating ttusb-budget/dspbootcode.bin
Generating cpia2/stv0672_vp4.bin
Generating av7110/bootcode.bin
make[2]: Leaving directory `/tmp/sandbox/media_build/v4l/firmware'
Kernel build directory is /tmp/sandbox/custom-linux-3.4
make -C ../linux apply_patches
make[2]: Entering directory `/tmp/sandbox/media_build/linux'
Patches for 3.4.3913 already applied.
make[2]: Leaving directory `/tmp/sandbox/media_build/linux'
make -C /tmp/sandbox/custom-linux-3.4 SUBDIRS=/tmp/sandbox/media_build/v4l modules make[2]: Entering directory `/tmp/sandbox/custom-linux-3.4'
CC [M] /tmp/sandbox/media_build/v4l/aptina-pll.o
In file included from <command-line>:0:0:
/tmp/sandbox/media_build/v4l/compat.h:1568:0: warning: "writel_relaxed" redefined
#define writel_relaxed writel
^
In file included from include/linux/scatterlist.h:10:0,

                 from /tmp/sandbox/media_build/v4l/compat.h:1255,
                 from <command-line>:0:

/tmp/sandbox/custom-linux-3.4/arch/arm/include/asm/io.h:235:0: note: this is the location of the previous definition
#define writel_relaxed(v,c) ((void)__raw_writel((__force u32) \
^
In file included from <command-line>:0:0:
/tmp/sandbox/media_build/v4l/compat.h: In function 'kvfree':
/tmp/sandbox/media_build/v4l/compat.h:1631:3: error: implicit declaration of function 'vfree' [-Werror=implicit-function-declaration]
vfree(addr);
^
cc1: some warnings being treated as errors
make[3]: *** [/tmp/sandbox/media_build/v4l/aptina-pll.o] Error 1
make[2]: *** [_module_/tmp/sandbox/media_build/v4l] Error 2
make[2]: Leaving directory `/tmp/sandbox/custom-linux-3.4'
make[1]: *** [default] Error 2
make[1]: Leaving directory `/tmp/sandbox/media_build/v4l'
make: *** [all] Error 2

Mauro Carvalho Chehab

2015-12-17 14:08 UTC #3 lore raw

Em Thu, 17 Dec 2015 14:52:21 +0100
Mason <[email protected]> escreveu:

Quoted 54 lines

Hello Mauro,
Quoted 14 lines · Mauro Carvalho Chehab
Quoted 12 lines · Mason
I have a TechnoTrend TT-TVStick CT2-4400v2 USB tuner, as described here:
http://linuxtv.org/wiki/index.php/TechnoTrend_TT-TVStick_CT2-4400
According to the article, the device is supported since kernel 3.19 and indeed, if I use a 4.1 kernel, I can pick CONFIG_DVB_USB_DVBSKY and everything seems to work.
Unfortunately (for me), I've been asked to make this driver work on an ancient 3.4 kernel.
The linuxtv article mentions:
Quoted 8 lines
Quoted 6 lines
"Drivers are included in kernel 3.17 (for version 1) and 3.19 (for version 2). They can be built with media_build for older kernels." ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This seems to imply that I can use the media_build framework to automatically (??) back-port a 3.19 driver to a 3.4 kernel?
"automatically" is a complex word ;)
If I get it working, I think you can even say "auto-magically" ;-)
Quoted 11 lines
Quoted 2 lines
This sounds too good to be true...
How far back can I go?
The goal is to allow compilation since 2.6.32, but please notice that not all drivers will go that far. Basically, when the backport seems too complex, we just remove the driver from the list of drivers that are compiled for a given legacy version.
Se the file v4l/versions.txt to double-check if the drivers you need have such restrictions. I suspect that, in the specific case of DVB_USB_DVBSKY, it should compile.
That is great news.
Quoted 5 lines
That doesn't mean that it was tested there. We don't test those backports to check against regressions. We only work, at best effort basis, to make them to build. So, use it with your own risk. If you find any problems, feel free to send us patches fixing it.
My first problem is that compilation fails on the first file ;-) See attached log.
My steps are:

Quoted 8 lines

cd media_build/linux
make tar DIR=/tmp/sandbox/media_tree
make untar
cd ..
make release DIR=/tmp/sandbox/custom-linux-3.4
make
I will investigate and report back.

Then I guess you're not using vanilla 3.4 Kernel, but some heavily modified version. You're on your own here.

Regards,
Mauro

Mason

author

2015-12-17 14:30 UTC #4 lore raw

Quoted 2 lines · Mauro Carvalho Chehab

Then I guess you're not using vanilla 3.4 Kernel, but some heavily modified version. You're on your own here.

#ifdef NEED_KVFREE
#include <linux/mm.h>
static inline void kvfree(const void *addr)

{
	if (is_vmalloc_addr(addr))
		vfree(addr);
	else
		kfree(addr);
}

#endif

/tmp/sandbox/media_build/v4l/compat.h: In function 'kvfree':
/tmp/sandbox/media_build/v4l/compat.h:1631:3: error: implicit declaration of function 'vfree' [-Werror=implicit-function-declaration]
vfree(addr);
^

vfree is declared in linux/vmalloc.h

The fix is trivial:

v4l/compat.h +1 -0

index c225c07d6caa..7f3f1d5f9d11 100644
@@ -1625,6 +1625,7 @@ static inline void eth_zero_addr(u8 *addr)
 
 #ifdef NEED_KVFREE
 #include <linux/mm.h>
+#include <linux/vmalloc.h>
 static inline void kvfree(const void *addr)
 {
        if (is_vmalloc_addr(addr))

Mauro Carvalho Chehab

2015-12-17 14:55 UTC #5 lore raw

Em Thu, 17 Dec 2015 15:30:43 +0100
Mason <[email protected]> escreveu:

Quoted 32 lines · Mauro Carvalho Chehab

Quoted 3 lines
Then I guess you're not using vanilla 3.4 Kernel, but some heavily modified version. You're on your own here.
#ifdef NEED_KVFREE
#include <linux/mm.h>
static inline void kvfree(const void *addr)
{
	if (is_vmalloc_addr(addr))
		vfree(addr);
	else
		kfree(addr);
}
#endif
/tmp/sandbox/media_build/v4l/compat.h: In function 'kvfree':
/tmp/sandbox/media_build/v4l/compat.h:1631:3: error: implicit declaration of function 'vfree' [-Werror=implicit-function-declaration]
   vfree(addr);
   ^
vfree is declared in linux/vmalloc.h
The fix is trivial:
v4l/compat.h +0 -0
index c225c07d6caa..7f3f1d5f9d11 100644

v4l/compat.h static inline void eth_zero_addr(u8 *addr) · line 1631

Quoted 7 lines

v4l/compat.h +1 -0

@@ -1625,6 +1625,7 @@ static inline void eth_zero_addr(u8 *addr)
 
 #ifdef NEED_KVFREE
 #include <linux/mm.h>
+#include <linux/vmalloc.h>
 static inline void kvfree(const void *addr)
 {
        if (is_vmalloc_addr(addr))

Well, it doesn't hurt to add it to the media_build tree, since vmalloc.h exists at least since 2.6.11.

Added upstream.

Did the driver compile fine?

Regards,
Mauro

Mason

author

2015-12-17 15:33 UTC #6 lore raw

Quoted 46 lines · Mauro Carvalho Chehab

Em Thu, 17 Dec 2015 15:30:43 +0100
Mason <[email protected]> escreveu:
Quoted 32 lines · Mauro Carvalho Chehab
Quoted 3 lines
Then I guess you're not using vanilla 3.4 Kernel, but some heavily modified version. You're on your own here.
#ifdef NEED_KVFREE
#include <linux/mm.h>
static inline void kvfree(const void *addr)
{
	if (is_vmalloc_addr(addr))
		vfree(addr);
	else
		kfree(addr);
}
#endif
/tmp/sandbox/media_build/v4l/compat.h: In function 'kvfree':
/tmp/sandbox/media_build/v4l/compat.h:1631:3: error: implicit declaration of function 'vfree' [-Werror=implicit-function-declaration]
   vfree(addr);
   ^
vfree is declared in linux/vmalloc.h
The fix is trivial:
v4l/compat.h +0 -0
index c225c07d6caa..7f3f1d5f9d11 100644
v4l/compat.h static inline void eth_zero_addr(u8 *addr) · line 1631
Quoted 7 lines
v4l/compat.h +1 -0
@@ -1625,6 +1625,7 @@ static inline void eth_zero_addr(u8 *addr)
 
 #ifdef NEED_KVFREE
 #include <linux/mm.h>
+#include <linux/vmalloc.h>
 static inline void kvfree(const void *addr)
 {
        if (is_vmalloc_addr(addr))
Well, it doesn't hurt to add it to the media_build tree, since vmalloc.h exists at least since 2.6.11.

Quoted 3 lines

Added upstream.
Did the driver compile fine?

I wanted to fix the NEED_WRITEL_RELAXED warning, but I don't know Perl.

v4l/scripts/make_config_compat.pl

check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h"); incorrectly outputs
#define NEED_WRITEL_RELAXED 1

In file included from <command-line>:0:0:
/tmp/sandbox/media_build/v4l/compat.h:1568:0: warning: "writel_relaxed" redefined
#define writel_relaxed writel
^
In file included from include/linux/scatterlist.h:10:0,

                 from /tmp/sandbox/media_build/v4l/compat.h:1255,
                 from <command-line>:0:

/tmp/sandbox/custom-linux-3.4/arch/arm/include/asm/io.h:235:0: note: this is the location of the previous definition
#define writel_relaxed(v,c) ((void)__raw_writel((__force u32) \
^

Shouldn't the script examine arch/$ARCH/include/asm/io.h instead of include/asm-generic/io.h ? (Or perhaps both?)

Does make_config_compat.pl know about ARCH?

The following patch makes "#define NEED_WRITEL_RELAXED 1" go away, but I'm looking for a general solution.

The next error is:

CC [M] /tmp/sandbox/media_build/v4l/dvb_net.o
/tmp/sandbox/media_build/v4l/dvb_net.c: In function 'dvb_net_add_if':
/tmp/sandbox/media_build/v4l/dvb_net.c:1244:38: error: macro "alloc_netdev" passed 4 arguments, but takes just 3

       NET_NAME_UNKNOWN, dvb_net_setup);
                                      ^
/tmp/sandbox/media_build/v4l/dvb_net.c:1243:8: error: 'alloc_netdev' undeclared (first use in this function)
  net = alloc_netdev(sizeof(struct dvb_net_priv), "dvb",
        ^

/tmp/sandbox/media_build/v4l/dvb_net.c:1243:8: note: each undeclared identifier is reported only once for each function it appears in /tmp/sandbox/media_build/v4l/dvb_net.c: At top level:
/tmp/sandbox/media_build/v4l/dvb_net.c:1205:13: warning: 'dvb_net_setup' defined but not used [-Wunused-function]
static void dvb_net_setup(struct net_device *dev)

Will look into it.

Regards.

v4l/scripts/make_config_compat.pl +1 -1

index 641f55e9c137..30a004525c08 100644
@@ -664,7 +664,7 @@ sub check_other_dependencies()
        check_files_for_func("DMA_ATTR_SKIP_CPU_SYNC", "NEED_DMA_ATTR_SKIP_CPU_SYNC", "include/linux/dma-attrs.h");
        check_files_for_func("sign_extend32", "NEED_SIGN_EXTEND32", "include/linux/bitops.h");
        check_files_for_func("netdev_dbg", "NEED_NETDEV_DBG", "include/linux/netdevice.h");
-       check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h");
+       check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "arch/arm/include/asm/io.h");
        check_files_for_func("get_user_pages_unlocked", "NEED_GET_USER_PAGES_UNLOCKED", "include/linux/mm.h");
        check_files_for_func("pr_warn_once", "NEED_PR_WARN_ONCE", "include/linux/printk.h");
        check_files_for_func("DIV_ROUND_CLOSEST_ULL", "NEED_DIV_ROUND_CLOSEST_ULL", "include/linux/kernel.h");

Mauro Carvalho Chehab

2015-12-17 16:09 UTC #7 lore raw

Em Thu, 17 Dec 2015 16:32:54 +0100
Mason <[email protected]> escreveu:

Quoted 68 lines · Mauro Carvalho Chehab

Quoted 47 lines
Em Thu, 17 Dec 2015 15:30:43 +0100
Mason <[email protected]> escreveu:
Quoted 32 lines · Mauro Carvalho Chehab
Quoted 3 lines
Then I guess you're not using vanilla 3.4 Kernel, but some heavily modified version. You're on your own here.
#ifdef NEED_KVFREE
#include <linux/mm.h>
static inline void kvfree(const void *addr)
{
	if (is_vmalloc_addr(addr))
		vfree(addr);
	else
		kfree(addr);
}
#endif
/tmp/sandbox/media_build/v4l/compat.h: In function 'kvfree':
/tmp/sandbox/media_build/v4l/compat.h:1631:3: error: implicit declaration of function 'vfree' [-Werror=implicit-function-declaration]
   vfree(addr);
   ^
vfree is declared in linux/vmalloc.h
The fix is trivial:
v4l/compat.h +0 -0
index c225c07d6caa..7f3f1d5f9d11 100644
v4l/compat.h static inline void eth_zero_addr(u8 *addr) · line 1631
Quoted 7 lines
v4l/compat.h +1 -0
@@ -1625,6 +1625,7 @@ static inline void eth_zero_addr(u8 *addr)
 
 #ifdef NEED_KVFREE
 #include <linux/mm.h>
+#include <linux/vmalloc.h>
 static inline void kvfree(const void *addr)
 {
        if (is_vmalloc_addr(addr))
Well, it doesn't hurt to add it to the media_build tree, since vmalloc.h exists at least since 2.6.11.
Quoted 3 lines
Added upstream.
Did the driver compile fine?
I wanted to fix the NEED_WRITEL_RELAXED warning, but I don't know Perl.
v4l/scripts/make_config_compat.pl
check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h"); incorrectly outputs
#define NEED_WRITEL_RELAXED 1
In file included from <command-line>:0:0:
/tmp/sandbox/media_build/v4l/compat.h:1568:0: warning: "writel_relaxed" redefined
#define writel_relaxed writel
^
In file included from include/linux/scatterlist.h:10:0,
                 from /tmp/sandbox/media_build/v4l/compat.h:1255,
                 from <command-line>:0:

Quoted 8 lines

/tmp/sandbox/custom-linux-3.4/arch/arm/include/asm/io.h:235:0: note: this is the location of the previous definition
#define writel_relaxed(v,c) ((void)__raw_writel((__force u32) \
^
Shouldn't the script examine arch/$ARCH/include/asm/io.h instead of include/asm-generic/io.h ? (Or perhaps both?)
Does make_config_compat.pl know about ARCH?

No to both. When you do a "make init" on the Kernel repository, it will evaluate the ARCH vars.

This is also needed for the media build to work, as it needs to check what CONFIG vars are enabled on the targeted Kernel.

Quoted 15 lines

The following patch makes "#define NEED_WRITEL_RELAXED 1" go away, but I'm looking for a general solution.
The next error is:
CC [M] /tmp/sandbox/media_build/v4l/dvb_net.o
/tmp/sandbox/media_build/v4l/dvb_net.c: In function 'dvb_net_add_if':
/tmp/sandbox/media_build/v4l/dvb_net.c:1244:38: error: macro "alloc_netdev" passed 4 arguments, but takes just 3
       NET_NAME_UNKNOWN, dvb_net_setup);
                                      ^
/tmp/sandbox/media_build/v4l/dvb_net.c:1243:8: error: 'alloc_netdev' undeclared (first use in this function)
  net = alloc_netdev(sizeof(struct dvb_net_priv), "dvb",
        ^

Quoted 6 lines

/tmp/sandbox/media_build/v4l/dvb_net.c:1243:8: note: each undeclared identifier is reported only once for each function it appears in /tmp/sandbox/media_build/v4l/dvb_net.c: At top level:
/tmp/sandbox/media_build/v4l/dvb_net.c:1205:13: warning: 'dvb_net_setup' defined but not used [-Wunused-function]
static void dvb_net_setup(struct net_device *dev)
Will look into it.

As I said before, heavily patched Kernel. It seems that the network stack was updated to some newer version. The media_build backport considers only the upstream Kernels. In the specific case of 3.4, it is known to build fine with Kernel linux-3.4.27. See:
http://hverkuil.home.xs4all.nl/logs/Wednesday.log

Regards,
Mauro

Quoted 19 lines

Regards.

v4l/scripts/make_config_compat.pl +1 -1

index 641f55e9c137..30a004525c08 100644
@@ -664,7 +664,7 @@ sub check_other_dependencies()
        check_files_for_func("DMA_ATTR_SKIP_CPU_SYNC", "NEED_DMA_ATTR_SKIP_CPU_SYNC", "include/linux/dma-attrs.h");
        check_files_for_func("sign_extend32", "NEED_SIGN_EXTEND32", "include/linux/bitops.h");
        check_files_for_func("netdev_dbg", "NEED_NETDEV_DBG", "include/linux/netdevice.h");
-       check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h");
+       check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "arch/arm/include/asm/io.h");
        check_files_for_func("get_user_pages_unlocked", "NEED_GET_USER_PAGES_UNLOCKED", "include/linux/mm.h");
        check_files_for_func("pr_warn_once", "NEED_PR_WARN_ONCE", "include/linux/printk.h");
        check_files_for_func("DIV_ROUND_CLOSEST_ULL", "NEED_DIV_ROUND_CLOSEST_ULL", "include/linux/kernel.h");

Mason

author

2015-12-17 16:49 UTC #8 lore raw

Quoted 26 lines · Mauro Carvalho Chehab

Em Thu, 17 Dec 2015 16:32:54 +0100
Mason <[email protected]> escreveu:
Quoted 23 lines
I wanted to fix the NEED_WRITEL_RELAXED warning, but I don't know Perl.
v4l/scripts/make_config_compat.pl
check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h"); incorrectly outputs
#define NEED_WRITEL_RELAXED 1
In file included from <command-line>:0:0:
/tmp/sandbox/media_build/v4l/compat.h:1568:0: warning: "writel_relaxed" redefined
#define writel_relaxed writel
^
In file included from include/linux/scatterlist.h:10:0,
                 from /tmp/sandbox/media_build/v4l/compat.h:1255,
                 from <command-line>:0:
/tmp/sandbox/custom-linux-3.4/arch/arm/include/asm/io.h:235:0: note: this is the location of the previous definition
#define writel_relaxed(v,c) ((void)__raw_writel((__force u32) \
^
Shouldn't the script examine arch/$ARCH/include/asm/io.h instead of include/asm-generic/io.h ? (Or perhaps both?)

Quoted 7 lines

Quoted 1 line
Does make_config_compat.pl know about ARCH?
No to both. When you do a "make init" on the Kernel repository, it will evaluate the ARCH vars.
This is also needed for the media build to work, as it needs to check what CONFIG vars are enabled on the targeted Kernel.

I downloaded the vanilla version of my custom kernel: linux-3.4.39.tar.xz

Even then, NEED_WRITEL_RELAXED is incorrectly defined.

How do you propose to fix this bug?

$ grep writel_relaxed arch/arm/include/asm/io.h
#define writel_relaxed(v,c)	((void)__raw_writel((__force u32) \
#define writel(v,c)		({ __iowmb(); writel_relaxed(v,c); })

$ grep writel_relaxed arch/x86/include/asm/io.h
$ grep -r writel_relaxed include

Quoted 5 lines

As I said before, heavily patched Kernel. It seems that the network stack was updated to some newer version. The media_build backport considers only the upstream Kernels. In the specific case of 3.4, it is known to build fine with Kernel linux-3.4.27. See:
http://hverkuil.home.xs4all.nl/logs/Wednesday.log

I will keep trying to get something to compile.

Regards.

Mauro Carvalho Chehab

2015-12-18 10:37 UTC #9 lore raw

Em Thu, 17 Dec 2015 17:48:57 +0100
Mason <[email protected]> escreveu:

Quoted 32 lines · Mauro Carvalho Chehab

Quoted 31 lines
Em Thu, 17 Dec 2015 16:32:54 +0100
Mason <[email protected]> escreveu:
Quoted 16 lines
I wanted to fix the NEED_WRITEL_RELAXED warning, but I don't know Perl.
v4l/scripts/make_config_compat.pl
check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h"); incorrectly outputs
#define NEED_WRITEL_RELAXED 1
In file included from <command-line>:0:0:
/tmp/sandbox/media_build/v4l/compat.h:1568:0: warning: "writel_relaxed" redefined
#define writel_relaxed writel
^
In file included from include/linux/scatterlist.h:10:0,
                 from /tmp/sandbox/media_build/v4l/compat.h:1255,
                 from <command-line>:0:
Quoted 8 lines
/tmp/sandbox/custom-linux-3.4/arch/arm/include/asm/io.h:235:0: note: this is the location of the previous definition
#define writel_relaxed(v,c) ((void)__raw_writel((__force u32) \
^
Shouldn't the script examine arch/$ARCH/include/asm/io.h instead of include/asm-generic/io.h ? (Or perhaps both?)
Does make_config_compat.pl know about ARCH?
No to both. When you do a "make init" on the Kernel repository, it will evaluate the ARCH vars.

Quoted 6 lines

Quoted 2 lines
This is also needed for the media build to work, as it needs to check what CONFIG vars are enabled on the targeted Kernel.
I downloaded the vanilla version of my custom kernel: linux-3.4.39.tar.xz
Even then, NEED_WRITEL_RELAXED is incorrectly defined.

did you run a:
make allmodconfig
make init

for the vanilla version? Without that, the symlinks won't appear.

Quoted 19 lines

How do you propose to fix this bug?
$ grep writel_relaxed arch/arm/include/asm/io.h
#define writel_relaxed(v,c)	((void)__raw_writel((__force u32) \
#define writel(v,c)		({ __iowmb(); writel_relaxed(v,c); })
$ grep writel_relaxed arch/x86/include/asm/io.h
$ grep -r writel_relaxed include
Quoted 5 lines
As I said before, heavily patched Kernel. It seems that the network stack was updated to some newer version. The media_build backport considers only the upstream Kernels. In the specific case of 3.4, it is known to build fine with Kernel linux-3.4.27. See:
http://hverkuil.home.xs4all.nl/logs/Wednesday.log
I will keep trying to get something to compile.
Regards.

Mason

author

2015-12-18 11:01 UTC #10 lore raw

Quoted 40 lines · Mauro Carvalho Chehab

Em Thu, 17 Dec 2015 17:48:57 +0100
Mason <[email protected]> escreveu:
Quoted 37 lines · Mauro Carvalho Chehab
Quoted 26 lines
Em Thu, 17 Dec 2015 16:32:54 +0100
Mason <[email protected]> escreveu:
Quoted 23 lines
I wanted to fix the NEED_WRITEL_RELAXED warning, but I don't know Perl.
v4l/scripts/make_config_compat.pl
check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h"); incorrectly outputs
#define NEED_WRITEL_RELAXED 1
In file included from <command-line>:0:0:
/tmp/sandbox/media_build/v4l/compat.h:1568:0: warning: "writel_relaxed" redefined
#define writel_relaxed writel
^
In file included from include/linux/scatterlist.h:10:0,
                 from /tmp/sandbox/media_build/v4l/compat.h:1255,
                 from <command-line>:0:
/tmp/sandbox/custom-linux-3.4/arch/arm/include/asm/io.h:235:0: note: this is the location of the previous definition
#define writel_relaxed(v,c) ((void)__raw_writel((__force u32) \
^
Shouldn't the script examine arch/$ARCH/include/asm/io.h instead of include/asm-generic/io.h ? (Or perhaps both?)
Quoted 7 lines
Quoted 1 line
Does make_config_compat.pl know about ARCH?
No to both. When you do a "make init" on the Kernel repository, it will evaluate the ARCH vars.
This is also needed for the media build to work, as it needs to check what CONFIG vars are enabled on the targeted Kernel.
I downloaded the vanilla version of my custom kernel: linux-3.4.39.tar.xz

Quoted 7 lines

Quoted 1 line
Even then, NEED_WRITEL_RELAXED is incorrectly defined.
did you run a:
make allmodconfig
make init
for the vanilla version? Without that, the symlinks won't appear.

/tmp/sandbox/linux-3.4.39$ make allmodconfig
scripts/kconfig/conf --allmodconfig Kconfig
#

# configuration written to .config
#
/tmp/sandbox/linux-3.4.39$ make menuconfig
scripts/kconfig/mconf Kconfig
.config:25:warning: symbol value '' invalid for PHYS_OFFSET
#
# configuration written to .config
#

*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.

/tmp/sandbox/linux-3.4.39$ make init scripts/kconfig/conf --silentoldconfig Kconfig CHK include/linux/version.h
CHK include/generated/utsrelease.h
make[1]: `include/generated/mach-types.h' is up to date. CC kernel/bounds.s
GEN include/generated/bounds.h
CC arch/arm/kernel/asm-offsets.s
GEN include/generated/asm-offsets.h
CALL scripts/checksyscalls.sh
CC scripts/mod/empty.o
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/file2alias.o
HOSTCC scripts/mod/modpost.o
HOSTCC scripts/mod/sumversion.o
HOSTLD scripts/mod/modpost
HOSTCC scripts/selinux/genheaders/genheaders HOSTCC scripts/selinux/mdp/mdp
CC init/main.o
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
CC init/do_mounts.o
CC init/do_mounts_initrd.o
LD init/mounts.o
CC init/initramfs.o
CC init/calibrate.o
LD init/built-in.o

cd ../media_build/linux
make tar DIR=/tmp/sandbox/media_tree
make untar
cd ..
make release DIR=/tmp/sandbox/linux-3.4.39
make

make -C ../linux apply_patches
make[2]: Entering directory `/tmp/sandbox/media_build/linux'
Patches for 3.4.39. already applied.
make[2]: Leaving directory `/tmp/sandbox/media_build/linux'
make -C /tmp/sandbox/linux-3.4.39 SUBDIRS=/tmp/sandbox/media_build/v4l modules make[2]: Entering directory `/tmp/sandbox/linux-3.4.39'
CC [M] /tmp/sandbox/media_build/v4l/altera-lpt.o
In file included from <command-line>:0:0:
/tmp/sandbox/media_build/v4l/compat.h:1568:0: warning: "writel_relaxed" redefined
#define writel_relaxed writel
^
In file included from include/linux/scatterlist.h:10:0,

                 from /tmp/sandbox/media_build/v4l/compat.h:1255,
                 from <command-line>:0:

/tmp/sandbox/linux-3.4.39/arch/arm/include/asm/io.h:235:0: note: this is the location of the previous definition
#define writel_relaxed(v,c) ((void)__raw_writel((__force u32) \
^

$ grep -rn NEED_WRITEL_RELAXED
v4l/compat.h:1567:#ifdef NEED_WRITEL_RELAXED
v4l/scripts/make_config_compat.pl:667:	check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h");
v4l/config-compat.h:1888:#define NEED_WRITEL_RELAXED 1

Mason

author

2015-12-17 17:03 UTC #11 lore raw

In reply to Mauro Carvalho Chehab

Quoted 5 lines · Mauro Carvalho Chehab

As I said before, heavily patched Kernel. It seems that the network stack was updated to some newer version. The media_build backport considers only the upstream Kernels. In the specific case of 3.4, it is known to build fine with Kernel linux-3.4.27. See:
http://hverkuil.home.xs4all.nl/logs/Wednesday.log

I don't think the network stack is different from vanilla...

I had a different idea:

The media_build process prints:

"Preparing to compile for kernel version 3.4.3913"

In fact, the custom kernel's Makefile contains:

VERSION = 3
PATCHLEVEL = 4
SUBLEVEL = 39
EXTRAVERSION = 13
NAME = Saber-toothed Squirrel

Is it possible that the build process gets confused by the EXTRAVERSION field?

Regards.

Mason

author

2015-12-17 17:17 UTC #12 lore raw

Quoted 18 lines · Mason

Quoted 5 lines · Mauro Carvalho Chehab
As I said before, heavily patched Kernel. It seems that the network stack was updated to some newer version. The media_build backport considers only the upstream Kernels. In the specific case of 3.4, it is known to build fine with Kernel linux-3.4.27. See:
http://hverkuil.home.xs4all.nl/logs/Wednesday.log
I don't think the network stack is different from vanilla...
I had a different idea:
The media_build process prints:
"Preparing to compile for kernel version 3.4.3913"
In fact, the custom kernel's Makefile contains:

Quoted 7 lines

VERSION = 3
PATCHLEVEL = 4
SUBLEVEL = 39
EXTRAVERSION = 13
NAME = Saber-toothed Squirrel
Is it possible that the build process gets confused by the EXTRAVERSION field?

Could this be the problem?
(Missing '.' between sublevel and extra)
Although with vanilla kernels, it will print 3.4.39. which is probably wrong...

v4l/Makefile +1 -1

index 1542092004fa..9147a98639b7 100644
@@ -233,9 +233,9 @@ ifneq ($(DIR),)
        -e '    elsif (/^EXTRAVERSION\s*=\s*(\S+)\n/){ $$extra=$$1; }' \
        -e '    elsif (/^KERNELSRC\s*:=\s*(\S.*)\n/ || /^MAKEARGS\s*:=\s*-C\s*(\S.*)\n/)' \
        -e '        { $$o=$$d; $$d=$$1; goto S; }' \
        -e '};' \
-       -e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s%s\n",' \
+       -e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s.%s\n",' \
        -e '    $$version,$$level,$$sublevel,$$version,$$level,$$sublevel,$$extra);' \
        -e 'print "OUTDIR:=$$o\n" if($$o);' \
        -e 'print "SRCDIR:=$$d\n";' > $(obj)/.version
        @cat .version|grep KERNELRELEASE:|sed s,'KERNELRELEASE:=','Forcing compiling to version ',

Mason

author

2015-12-18 10:44 UTC #13 lore raw

In reply to Mason

Quoted 6 lines · Mason

The media_build process prints:
"Preparing to compile for kernel version 3.4.3913"
In fact, the custom kernel's Makefile contains:

Quoted 7 lines

VERSION = 3
PATCHLEVEL = 4
SUBLEVEL = 39
EXTRAVERSION = 13
NAME = Saber-toothed Squirrel
Is it possible that the build process gets confused by the EXTRAVERSION field?

Here's the problem:

v4l/Makefile writes to KERNELRELEASE and v4l/.version

	-e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s%s\n",' \
	-e '	$$version,$$level,$$sublevel,$$version,$$level,$$sublevel,$$extra);' \

$ cat v4l/.version 
VERSION=3
PATCHLEVEL:=4
SUBLEVEL:=39
KERNELRELEASE:=3.4.3913
SRCDIR:=/tmp/sandbox/custom-linux-3.4

Then $(MAKE) -C ../linux apply_patches calls
patches_for_kernel.pl 3.4.3913

which computes kernel_version
= 3 << 16 + 4 << 8 + 3913 = 0x031349

which is incorrectly interpreted as kernel 3.19.73 thus the correct patches are not applied.

Trivial patch follows. Will test right away.

Regards.

v4l/Makefile +1 -1

index 1542092004fa..9147a98639b7 100644
@@ -233,9 +233,9 @@ ifneq ($(DIR),)
        -e '    elsif (/^EXTRAVERSION\s*=\s*(\S+)\n/){ $$extra=$$1; }' \
        -e '    elsif (/^KERNELSRC\s*:=\s*(\S.*)\n/ || /^MAKEARGS\s*:=\s*-C\s*(\S.*)\n/)' \
        -e '        { $$o=$$d; $$d=$$1; goto S; }' \
        -e '};' \
-       -e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s%s\n",' \
+       -e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s.%s\n",' \
        -e '    $$version,$$level,$$sublevel,$$version,$$level,$$sublevel,$$extra);' \
        -e 'print "OUTDIR:=$$o\n" if($$o);' \
        -e 'print "SRCDIR:=$$d\n";' > $(obj)/.version
        @cat .version|grep KERNELRELEASE:|sed s,'KERNELRELEASE:=','Forcing compiling to version ',

Mauro Carvalho Chehab

2015-12-18 11:03 UTC #14 lore raw

Em Fri, 18 Dec 2015 11:44:35 +0100
Mason <[email protected]> escreveu:

Quoted 48 lines · Mason

Quoted 7 lines
The media_build process prints:
"Preparing to compile for kernel version 3.4.3913"
In fact, the custom kernel's Makefile contains:
Quoted 7 lines
VERSION = 3
PATCHLEVEL = 4
SUBLEVEL = 39
EXTRAVERSION = 13
NAME = Saber-toothed Squirrel
Is it possible that the build process gets confused by the EXTRAVERSION field?
Here's the problem:
v4l/Makefile writes to KERNELRELEASE and v4l/.version
	-e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s%s\n",' \
	-e '	$$version,$$level,$$sublevel,$$version,$$level,$$sublevel,$$extra);' \
$ cat v4l/.version 
VERSION=3
PATCHLEVEL:=4
SUBLEVEL:=39
KERNELRELEASE:=3.4.3913
SRCDIR:=/tmp/sandbox/custom-linux-3.4
Then $(MAKE) -C ../linux apply_patches calls
patches_for_kernel.pl 3.4.3913
which computes kernel_version
= 3 << 16 + 4 << 8 + 3913 = 0x031349
which is incorrectly interpreted as kernel 3.19.73 thus the correct patches are not applied.
Trivial patch follows. Will test right away.
Regards.
v4l/Makefile +0 -0
index 1542092004fa..9147a98639b7 100644

v4l/Makefile ifneq ($(DIR),) · line 238

Quoted 7 lines

v4l/Makefile +1 -1

@@ -233,9 +233,9 @@ ifneq ($(DIR),)
        -e '    elsif (/^EXTRAVERSION\s*=\s*(\S+)\n/){ $$extra=$$1; }' \
        -e '    elsif (/^KERNELSRC\s*:=\s*(\S.*)\n/ || /^MAKEARGS\s*:=\s*-C\s*(\S.*)\n/)' \
        -e '        { $$o=$$d; $$d=$$1; goto S; }' \
        -e '};' \
-       -e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s%s\n",' \
+       -e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s.%s\n",' \
        -e '    $$version,$$level,$$sublevel,$$version,$$level,$$sublevel,$$extra);' \

Hmm... that doesn't sound right on upstream Kernels.

For example, the extra version on the media_build current Kernel is:

Makefile:EXTRAVERSION = -rc2

So, I guess we'll need a different regex, like:

         -e '    elsif (/^EXTRAVERSION\s*=\s*(\d+)\n/){ $$extra=".$$1"; }' \
         -e '    elsif (/^EXTRAVERSION\s*=\s*(\S+)\n/){ $$extra=$$1; }' \

Regards,
Mauro

Mauro Carvalho Chehab

2015-12-18 11:22 UTC #15 lore raw

Em Fri, 18 Dec 2015 09:03:45 -0200
Mauro Carvalho Chehab <[email protected]> escreveu:

Quoted 61 lines

Em Fri, 18 Dec 2015 11:44:35 +0100
Mason <[email protected]> escreveu:
Quoted 48 lines · Mason
Quoted 7 lines
The media_build process prints:
"Preparing to compile for kernel version 3.4.3913"
In fact, the custom kernel's Makefile contains:
Quoted 7 lines
VERSION = 3
PATCHLEVEL = 4
SUBLEVEL = 39
EXTRAVERSION = 13
NAME = Saber-toothed Squirrel
Is it possible that the build process gets confused by the EXTRAVERSION field?
Here's the problem:
v4l/Makefile writes to KERNELRELEASE and v4l/.version
	-e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s%s\n",' \
	-e '	$$version,$$level,$$sublevel,$$version,$$level,$$sublevel,$$extra);' \
$ cat v4l/.version 
VERSION=3
PATCHLEVEL:=4
SUBLEVEL:=39
KERNELRELEASE:=3.4.3913
SRCDIR:=/tmp/sandbox/custom-linux-3.4
Then $(MAKE) -C ../linux apply_patches calls
patches_for_kernel.pl 3.4.3913
which computes kernel_version
= 3 << 16 + 4 << 8 + 3913 = 0x031349
which is incorrectly interpreted as kernel 3.19.73 thus the correct patches are not applied.
Trivial patch follows. Will test right away.
Regards.
v4l/Makefile +0 -0
index 1542092004fa..9147a98639b7 100644
v4l/Makefile ifneq ($(DIR),) · line 238
Quoted 7 lines
v4l/Makefile +1 -1
@@ -233,9 +233,9 @@ ifneq ($(DIR),)
        -e '    elsif (/^EXTRAVERSION\s*=\s*(\S+)\n/){ $$extra=$$1; }' \
        -e '    elsif (/^KERNELSRC\s*:=\s*(\S.*)\n/ || /^MAKEARGS\s*:=\s*-C\s*(\S.*)\n/)' \
        -e '        { $$o=$$d; $$d=$$1; goto S; }' \
        -e '};' \
-       -e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s%s\n",' \
+       -e 'printf ("VERSION=%s\nPATCHLEVEL:=%s\nSUBLEVEL:=%s\nKERNELRELEASE:=%s.%s.%s.%s\n",' \
        -e '    $$version,$$level,$$sublevel,$$version,$$level,$$sublevel,$$extra);' \
Hmm... that doesn't sound right on upstream Kernels.

Quoted 8 lines

For example, the extra version on the media_build current Kernel is:
Makefile:EXTRAVERSION = -rc2
So, I guess we'll need a different regex, like:
         -e '    elsif (/^EXTRAVERSION\s*=\s*(\d+)\n/){ $$extra=".$$1"; }' \
         -e '    elsif (/^EXTRAVERSION\s*=\s*(\S+)\n/){ $$extra=$$1; }' \

Yes, this works. Changing a 2.6.32 kernel to:

VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 32
EXTRAVERSION = 99

It gets:
Forcing compiling to version 2.6.32.99 make[1]: Leaving directory '/devel/v4l/media_build/v4l'

And

	KERNELRELEASE:=2.6.32.99

Changing EXTRAVERSION to:
EXTRAVERSION = -rc99

It gets:
Forcing compiling to version 2.6.32-rc99
And

	 KERNELRELEASE:=2.6.32-rc99

Patch applied.

Regards,
Mauro

Mason

author

2015-12-18 12:11 UTC #16 lore raw

Quoted 1 line · Mauro Carvalho Chehab

Patch applied.

Great! Thanks.

Using the latest media_build master + my writel_relaxed work-around, compilation proceeds much further, then dies on device tree stuff:
(same error with vanilla and custom kernel)

Will look into it. Any idea? :-(

By the way, if I was not clear, I'm cross-compiling for an ARM platform.

CC [M] /tmp/sandbox/media_build/v4l/v4l2-of.o
/tmp/sandbox/media_build/v4l/v4l2-of.c: In function 'v4l2_of_parse_csi_bus':
/tmp/sandbox/media_build/v4l/v4l2-of.c:38:4: error: implicit declaration of function 'of_prop_next_u32' [-Werror=implicit-function-declaration]

    lane = of_prop_next_u32(prop, lane, &v);
    ^
/tmp/sandbox/media_build/v4l/v4l2-of.c:38:9: warning: assignment makes pointer from integer without a cast
    lane = of_prop_next_u32(prop, lane, &v);
         ^
/tmp/sandbox/media_build/v4l/v4l2-of.c:52:13: warning: assignment makes pointer from integer without a cast
    polarity = of_prop_next_u32(prop, polarity, &v);
             ^
/tmp/sandbox/media_build/v4l/v4l2-of.c: In function 'v4l2_of_parse_link':
/tmp/sandbox/media_build/v4l/v4l2-of.c:287:24: warning: passing argument 1 of 'of_parse_phandle' discards 'const' qualifier from pointer target type
  np = of_parse_phandle(node, "remote-endpoint", 0);
                        ^
In file included from include/linux/i2c.h:36:0,
                 from /tmp/sandbox/media_build/v4l/compat.h:977,
                 from <command-line>:0:
include/linux/of.h:237:28: note: expected 'struct device_node *' but argument is of type 'const struct device_node *'
 extern struct device_node *of_parse_phandle(struct device_node *np,
                            ^

cc1: some warnings being treated as errors
make[3]: *** [/tmp/sandbox/media_build/v4l/v4l2-of.o] Error 1 make[2]: *** [_module_/tmp/sandbox/media_build/v4l] Error 2 make[2]: Leaving directory `/tmp/sandbox/linux-3.4.39' make[1]: *** [default] Error 2
make[1]: Leaving directory `/tmp/sandbox/media_build/v4l'
make: *** [all] Error 2

Mason

author

2015-12-18 12:59 UTC #17 lore raw

Quoted 12 lines · Mason

Quoted 1 line · Mauro Carvalho Chehab
Patch applied.
Great! Thanks.
Using the latest media_build master + my writel_relaxed work-around, compilation proceeds much further, then dies on device tree stuff:
(same error with vanilla and custom kernel)
Will look into it. Any idea? :-(

Quoted 7 lines

By the way, if I was not clear, I'm cross-compiling for an ARM platform.
CC [M] /tmp/sandbox/media_build/v4l/v4l2-of.o
/tmp/sandbox/media_build/v4l/v4l2-of.c: In function 'v4l2_of_parse_csi_bus':
/tmp/sandbox/media_build/v4l/v4l2-of.c:38:4: error: implicit declaration of function 'of_prop_next_u32' [-Werror=implicit-function-declaration]
    lane = of_prop_next_u32(prop, lane, &v);
    ^

of_prop_next_u32() was introduced by commit c541adc637066

$ git describe --contains c541adc637066
v3.5-rc1~176^2~34

So it seems something needs to be done for kernels older than 3.5

I'll hack around it by adding

static inline const __be32 *of_prop_next_u32(struct property *prop,

		const __be32 *cur, u32 *pu)
{
	return NULL;
}

What's the correct fix?

Quoted 4 lines

/tmp/sandbox/media_build/v4l/v4l2-of.c: In function 'v4l2_of_parse_link':
/tmp/sandbox/media_build/v4l/v4l2-of.c:287:24: warning: passing argument 1 of 'of_parse_phandle' discards 'const' qualifier from pointer target type
  np = of_parse_phandle(node, "remote-endpoint", 0);
                        ^

Commit b8fbdc42c5c5d made the first parameter const.

$ git describe --contains b8fbdc42c5c5d
v3.8-rc1~105^2~13

I suppose I can live with the warning for now.

Regards.

Mason

author

2015-12-18 13:40 UTC #18 lore raw

On 18/12/2015 13:59, Mason wrote: [snip previous work-arounds]

Compilation completes.

make -C /tmp/sandbox/custom-linux-3.4 SUBDIRS=/tmp/sandbox/media_build/v4l modules make[2]: Entering directory `/tmp/sandbox/custom-linux-3.4'
Building modules, stage 2.
MODPOST 209 modules
WARNING: "of_graph_parse_endpoint" [/tmp/sandbox/media_build/v4l/videodev.ko] undefined!
WARNING: "of_get_next_parent" [/tmp/sandbox/media_build/v4l/videodev.ko] undefined!
WARNING: "nsecs_to_jiffies" [/tmp/sandbox/media_build/v4l/gpio-ir-recv.ko] undefined! make[2]: Leaving directory `/tmp/sandbox/custom-linux-3.4' ./scripts/rmmod.pl check
found 209 modules
make[1]: Leaving directory `/tmp/sandbox/media_build/v4l'

A few link problems, two from device tree:

of_graph_parse_endpoint() commit fd9fdb78a9bf8

of_get_next_parent() was not exported until commit 6695be6863b75

nsecs_to_jiffies() was not exported until commit d560fed6abe0f

How would you fix those?

I will try building a kernel with CONFIG_OF=n

Regards.

Mason

author

2015-12-18 15:13 UTC #19 lore raw

Quoted 1 line · Mason

I will try building a kernel with CONFIG_OF=n

Build works much better with CONFIG_OF=n
I suppose OF support with ancient kernels is untested. (My setup didn't need it anyway.)

The remaining issue is:

WARNING: "nsecs_to_jiffies" [/tmp/sandbox/media_build/v4l/gpio-ir-recv.ko] undefined!

$ git describe --contains d560fed6abe0f
v3.17-rc1~109^2~40

The actual call site was added recently by commit 3fb136f3392d (Hasn't even it linux-stable yet, I only see it in next-20151123)

I think a patch is needed for kernels < 3.17 right?

Regards.

Mason

author

2015-12-18 17:10 UTC #20 lore raw

In reply to Mauro Carvalho Chehab

Quoted 1 line · Mauro Carvalho Chehab

Did the driver compile fine?

Once all the modules finish building, I have many *.ko files in v4l.

I copy them to my target's /lib/modules/3.4.39.13/kernel and run depmod -a

It seems modprobe rc-dvbsky doesn't bring in all the stuff needed to actually use the device.

I added a few more modules at random:

Module                  Size  Used by
dvb_core               73479  0 
v4l2_common             1628  0 
videodev              111886  1 v4l2_common
media                   9838  1 videodev
rc_dvbsky                815  0 
rc_core                15521  1 rc_dvbsky

Probably needs dvb_usb but I don't see it built...

$ grep -rn '\<CONFIG_DVB_USB\>'
linux/drivers/media/usb/dvb-usb/Makefile:3:obj-$(CONFIG_DVB_USB) += dvb-usb.o
v4l/.myconfig:245:CONFIG_DVB_USB                               := n
v4l/Makefile.media:1010:obj-$(CONFIG_DVB_USB) += dvb-usb.o
v4l/config-compat.h:548:#undef CONFIG_DVB_USB
v4l/.config:183:# CONFIG_DVB_USB is not set

(All these files are generated by the build process.)

Am I doing something wrong?

Regards.

Mason

author

2015-12-18 19:42 UTC #21 lore raw

Quoted 1 line · Mason

Am I doing something wrong?

Yes, I didn't have I2C enabled.

By running 'make menuconfig' in a recent kernel, I could see which Kconfig options are required to build the dvbsky driver.

Mauro, I hope you'll find time to address my remaining bug reports.

Issue #1
check_files_for_func("writel_relaxed", "NEED_WRITEL_RELAXED", "include/asm-generic/io.h"); is definitely incorrect for older kernels.

Issue #2
WARNING: "nsecs_to_jiffies" [/tmp/sandbox/media_build/v4l/gpio-ir-recv.ko] undefined!

Regards.

Mason

author

2015-12-28 10:44 UTC #22 lore raw

In reply to Mauro Carvalho Chehab

Hello Mauro,

Haven't heard back from you in a while. Maybe someone else can point out what I'm doing wrong?

Quoted 12 lines · Mauro Carvalho Chehab

Quoted 9 lines · Mason
I have a TechnoTrend TT-TVStick CT2-4400v2 USB tuner, as described here:
http://linuxtv.org/wiki/index.php/TechnoTrend_TT-TVStick_CT2-4400
According to the article, the device is supported since kernel 3.19 and indeed, if I use a 4.1 kernel, I can pick CONFIG_DVB_USB_DVBSKY and everything seems to work.
Unfortunately (for me), I've been asked to make this driver work on an ancient 3.4 kernel.

Quoted 8 lines

The goal is to allow compilation since 2.6.32, but please notice that not all drivers will go that far. Basically, when the backport seems too complex, we just remove the driver from the list of drivers that are compiled for a given legacy version.
See the file v4l/versions.txt to double-check if the drivers you need have such restrictions. I suspect that, in the specific case of DVB_USB_DVBSKY, it should compile.

Whatever options I pick for my 3.4 config, CONFIG_DVB_USB_DVBSKY remains unset in v4l/.config

$ grep -r DVB_USB_DVBSKY media_build/v4l/
media_build/v4l/Kconfig:config DVB_USB_DVBSKY
media_build/v4l/Kconfig.kern: [snip config USB]
media_build/v4l/Kconfig.kern: [snip config I2C]
media_build/v4l/.myconfig:CONFIG_DVB_USB_DVBSKY                        := n
media_build/v4l/Makefile.media:obj-$(CONFIG_DVB_USB_DVBSKY) += dvb-usb-dvbsky.o
media_build/v4l/.config:# CONFIG_DVB_USB_DVBSKY is not set

I suppose some prerequisite is missing?
Does anything obvious come to mind?

I've resorted to interrupting the build and changing v4l/.config to CONFIG_DVB_USB_DVBSKY=m (and the module is correctly built) but this feels like an unnecessary hack.

Regards.

Mason

author

2015-12-28 12:21 UTC #23 lore raw

Quoted 39 lines · Mason

Hello Mauro,
Haven't heard back from you in a while. Maybe someone else can point out what I'm doing wrong?
Quoted 12 lines · Mauro Carvalho Chehab
Quoted 9 lines · Mason
I have a TechnoTrend TT-TVStick CT2-4400v2 USB tuner, as described here:
http://linuxtv.org/wiki/index.php/TechnoTrend_TT-TVStick_CT2-4400
According to the article, the device is supported since kernel 3.19 and indeed, if I use a 4.1 kernel, I can pick CONFIG_DVB_USB_DVBSKY and everything seems to work.
Unfortunately (for me), I've been asked to make this driver work on an ancient 3.4 kernel.
Quoted 8 lines
The goal is to allow compilation since 2.6.32, but please notice that not all drivers will go that far. Basically, when the backport seems too complex, we just remove the driver from the list of drivers that are compiled for a given legacy version.
See the file v4l/versions.txt to double-check if the drivers you need have such restrictions. I suspect that, in the specific case of DVB_USB_DVBSKY, it should compile.
Whatever options I pick for my 3.4 config, CONFIG_DVB_USB_DVBSKY remains unset in v4l/.config
$ grep -r DVB_USB_DVBSKY media_build/v4l/
media_build/v4l/Kconfig:config DVB_USB_DVBSKY
media_build/v4l/Kconfig.kern: [snip config USB]
media_build/v4l/Kconfig.kern: [snip config I2C]
media_build/v4l/.myconfig:CONFIG_DVB_USB_DVBSKY                        := n
media_build/v4l/Makefile.media:obj-$(CONFIG_DVB_USB_DVBSKY) += dvb-usb-dvbsky.o
media_build/v4l/.config:# CONFIG_DVB_USB_DVBSKY is not set

Quoted 6 lines

I suppose some prerequisite is missing?
Does anything obvious come to mind?
I've resorted to interrupting the build and changing v4l/.config to CONFIG_DVB_USB_DVBSKY=m (and the module is correctly built) but this feels like an unnecessary hack.

/tmp/sandbox/media_build$ make allmodconfig

didn't add anything on top of what the vanilla 'make' did.

$ make menuconfig
make -C /tmp/sandbox/media_build/v4l menuconfig
make[1]: Entering directory `/tmp/sandbox/media_build/v4l'
/tmp/buildroot-2014.05-13/output/build/linux-custom/scripts/kconfig/mconf ./Kconfig
./Kconfig:519: syntax error
./Kconfig:518: unknown option "Say"
./Kconfig:519: unknown option "To"
./Kconfig:520: unknown option "called"
./Kconfig:523: syntax error
./Kconfig:522:warning: multi-line strings not supported
./Kconfig:522: unknown option "If"
make[1]: *** [menuconfig] Error 1
make[1]: Leaving directory `/tmp/sandbox/media_build/v4l'
make: *** [menuconfig] Error 2

I'll keep poking random knobs.

Regards.

Mauro Carvalho Chehab

2015-12-28 13:02 UTC #24 lore raw

Hi Mason,

Em Mon, 28 Dec 2015 13:21:03 +0100
Mason <[email protected]> escreveu:

Quoted 67 lines · Mason

Quoted 40 lines
Hello Mauro,
Haven't heard back from you in a while. Maybe someone else can point out what I'm doing wrong?
Quoted 12 lines · Mauro Carvalho Chehab
Quoted 9 lines · Mason
I have a TechnoTrend TT-TVStick CT2-4400v2 USB tuner, as described here:
http://linuxtv.org/wiki/index.php/TechnoTrend_TT-TVStick_CT2-4400
According to the article, the device is supported since kernel 3.19 and indeed, if I use a 4.1 kernel, I can pick CONFIG_DVB_USB_DVBSKY and everything seems to work.
Unfortunately (for me), I've been asked to make this driver work on an ancient 3.4 kernel.
Quoted 8 lines
The goal is to allow compilation since 2.6.32, but please notice that not all drivers will go that far. Basically, when the backport seems too complex, we just remove the driver from the list of drivers that are compiled for a given legacy version.
See the file v4l/versions.txt to double-check if the drivers you need have such restrictions. I suspect that, in the specific case of DVB_USB_DVBSKY, it should compile.
Whatever options I pick for my 3.4 config, CONFIG_DVB_USB_DVBSKY remains unset in v4l/.config
$ grep -r DVB_USB_DVBSKY media_build/v4l/
media_build/v4l/Kconfig:config DVB_USB_DVBSKY
media_build/v4l/Kconfig.kern: [snip config USB]
media_build/v4l/Kconfig.kern: [snip config I2C]
media_build/v4l/.myconfig:CONFIG_DVB_USB_DVBSKY                        := n
media_build/v4l/Makefile.media:obj-$(CONFIG_DVB_USB_DVBSKY) += dvb-usb-dvbsky.o
media_build/v4l/.config:# CONFIG_DVB_USB_DVBSKY is not set
Quoted 6 lines
I suppose some prerequisite is missing?
Does anything obvious come to mind?
I've resorted to interrupting the build and changing v4l/.config to CONFIG_DVB_USB_DVBSKY=m (and the module is correctly built) but this feels like an unnecessary hack.
/tmp/sandbox/media_build$ make allmodconfig
didn't add anything on top of what the vanilla 'make' did.
$ make menuconfig
make -C /tmp/sandbox/media_build/v4l menuconfig
make[1]: Entering directory `/tmp/sandbox/media_build/v4l'
/tmp/buildroot-2014.05-13/output/build/linux-custom/scripts/kconfig/mconf ./Kconfig
./Kconfig:519: syntax error
./Kconfig:518: unknown option "Say"
./Kconfig:519: unknown option "To"
./Kconfig:520: unknown option "called"
./Kconfig:523: syntax error
./Kconfig:522:warning: multi-line strings not supported
./Kconfig:522: unknown option "If"
make[1]: *** [menuconfig] Error 1
make[1]: Leaving directory `/tmp/sandbox/media_build/v4l'
make: *** [menuconfig] Error 2

Quoted 1 line

I'll keep poking random knobs.

The maintainance of the media_build tree is at best effort basis, and we really don't have much time to fix things there. Personally, I never use it, as all the tests I do are with the latest Kernel nowadays.

I think Hans use it for some of his tests, but, from his daily results [1], the current tree works for him. So, it is unlikely that he'll be able to reproduce the issues you're reporting.

[1] see the e-mails with "cron job: media_tree daily build:" on the subject.

So, instead of pointing the issues, the best you can do is to send your fixup patches. if they won't break for the tested compilation scenario, they'll be applied.

Yet, that may take some time for us to apply, as Hans is in vacations, and I'm planning to take some days off in Jan.

Regards,
Mauro