media: dvb-core: add upper bound check in DMX_SET_BUFFER_SIZE ioctl
PATCH 1/1 media: dvb-core: add upper bound check in DMX_SET_BUFFER_SIZE ioctl
dvb_dvr_set_buffer_size() and dvb_dmxdev_set_buffer_size() pass the user-supplied size argument directly to vmalloc() without any upper bound check. This allows excessive kernel memory allocation via the DMX_SET_BUFFER_SIZE ioctl, which can lead to system-wide OOM conditions.
Kernel panic - not syncing: System is deadlocked on memory Call Trace: out_of_memory+0x12fd/0x1370 __alloc_frozen_pages_noprof+0x2620/0x2fa0 __vmalloc_node_range_noprof+0x7fa/0x1490 dvb_dvr_do_ioctl+0x11e/0x260 (drivers/media/dvb-core/dmxdev.c:296) dvb_usercopy+0x15b/0x360
Fix by cap both functions at 64 MB and return -EINVAL for oversized requests.
--- drivers/media/dvb-core/dmxdev.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/media/dvb-core/dmxdev.c +7 -0
@@ -20,6 +20,9 @@#include <media/dmxdev.h> #include <media/dvb_vb2.h> +/* 64 MB upper bound for DVB ring buffer allocations */ +#define DVB_BUFFER_SIZE_MAX (64 * 1024 * 1024) +static int debug;module_param (debug, int,0644 );
@@ -292,6 +295,8 @@ static int dvb_dvr_set_buffer_size(struct dmxdev *dmxdev,return 0 ;if (!size)return -EINVAL ; +if (size >DVB_BUFFER_SIZE_MAX ) +return -EINVAL ; newmem =vmalloc (size);if (!newmem)
@@ -333,6 +338,8 @@ static int dvb_dmxdev_set_buffer_size(struct dmxdev_filter *dmxdevfilter,return -EINVAL ;if (dmxdevfilter->state >=DMXDEV_STATE_GO )return -EBUSY ; +if (size >DVB_BUFFER_SIZE_MAX ) +return -EINVAL ; newmem =vmalloc (size);if (!newmem)
Signature · 1 line
2.53.0
Quoted 30 lines · On Wed, 22 Jul 2026 15:37:19 -0400 "Cen Zhang (Microsoft)" <[email protected]>…
dvb_dvr_set_buffer_size()anddvb_dmxdev_set_buffer_size()pass the user-supplied size argument directly tovmalloc()without any upper bound check. This allows excessive kernel memory allocation via theDMX_SET_BUFFER_SIZEioctl, which can lead to system-wide OOM conditions.Kernel panic - not syncing: System is deadlocked on memory Call Trace: out_of_memory+0x12fd/0x1370 __alloc_frozen_pages_noprof+0x2620/0x2fa0 __vmalloc_node_range_noprof+0x7fa/0x1490 dvb_dvr_do_ioctl+0x11e/0x260 (drivers/media/dvb-core/dmxdev.c:296) dvb_usercopy+0x15b/0x360Fix by cap both functions at 64 MB and return
-EINVALfor oversized requests.Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Fixes a095be4b030c ("V4L/DVB (7659): dvb-core: Implement DMX_SET_BUFFER_SIZE for dvr")Reported-by [email protected]Signed-off-by Cen Zhang (Microsoft) <[email protected]>--- drivers/media/dvb-core/dmxdev.c | 7 +++++++ 1 file changed, 7 insertions(+)drivers/media/dvb-core/dmxdev.c +0
-0
drivers/media/dvb-core/dmxdev.c line 24
Quoted 5 lines
drivers/media/dvb-core/dmxdev.c +2
-0@@ -20,6 +20,9 @@#include <media/dmxdev.h> #include <media/dvb_vb2.h> +/* 64 MB upper bound for DVB ring buffer allocations */ +#define DVB_BUFFER_SIZE_MAX (64 * 1024 * 1024)
Having a limit is good, but why 64MB?
Can you provide me more details about the test scenario: e.g. with what TV standards had you test it, and such.
Regards,
Mauro
Thanks,
Mauro
Quoted 36 lines · On Wed, 29 Jul 2026 08:30:26 +0200 Mauro Carvalho Chehab…
Quoted 33 lines · On Wed, 22 Jul 2026 15:37:19 -0400 "Cen Zhang (Microsoft)" <[email protected]>…
dvb_dvr_set_buffer_size()anddvb_dmxdev_set_buffer_size()pass the user-supplied size argument directly tovmalloc()without any upper bound check. This allows excessive kernel memory allocation via theDMX_SET_BUFFER_SIZEioctl, which can lead to system-wide OOM conditions.Kernel panic - not syncing: System is deadlocked on memory Call Trace: out_of_memory+0x12fd/0x1370 __alloc_frozen_pages_noprof+0x2620/0x2fa0 __vmalloc_node_range_noprof+0x7fa/0x1490 dvb_dvr_do_ioctl+0x11e/0x260 (drivers/media/dvb-core/dmxdev.c:296) dvb_usercopy+0x15b/0x360Fix by cap both functions at 64 MB and return
-EINVALfor oversized requests.Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Fixes a095be4b030c ("V4L/DVB (7659): dvb-core: Implement DMX_SET_BUFFER_SIZE for dvr")Reported-by [email protected]Signed-off-by Cen Zhang (Microsoft) <[email protected]>--- drivers/media/dvb-core/dmxdev.c | 7 +++++++ 1 file changed, 7 insertions(+)drivers/media/dvb-core/dmxdev.c +0
-0@@ -20,6 +20,9 @@#include <media/dmxdev.h> #include <media/dvb_vb2.h>
Quoted 7 lines
drivers/media/dvb-core/dmxdev.c line 24
Quoted 2 lines
drivers/media/dvb-core/dmxdev.c +2
-0+/* 64 MB upper bound for DVB ring buffer allocations */ +#define DVB_BUFFER_SIZE_MAX (64 * 1024 * 1024)Having a limit is good, but why 64MB?
Can you provide me more details about the test scenario: e.g. with what TV standards had you test it, and such.
Forgot to mention, but instead of returning -EINVAL, probably the best would be to setup the ringbuffer size to the maximum value, as otherwise this would break existing apps.
Btw, what apps did you use to test it? Had you check what's the current limit on what apps?
Regards,
Mauro
Hi Mauro,
Quoted 3 lines
Forgot to mention, but instead of returning
-EINVAL, probably the best would be to setup the ringbuffer size to the maximum value, as otherwise this would break existing apps.
Thanks for pointing this out. I will revise this in v3.
Quoted 4 lines
Having a limit is good, but why 64MB?
Btw, what apps did you use to test it? Had you check what's the current limit on what apps?
I chose 64 MiB as a conservative upper bound based on a brief source survey of DVB applications:
- The kernel DVR buffer defaults to approximately 1.84 MiB. In the upstream revisions I checked, TVheadend, MythTV and MuMuDVB do not actively call DMX_SET_BUFFER_SIZE and therefore retain this default:
- SATPI allows approximately 30 MiB:
- DVBlast defaults to approximately 7.34 MiB:
https://github.com/videolan/dvblast/blob/ae6b24ae8cdf6eaa1ada8534a257258f3c906a77/dvb.c#L71
- dvbv5-zap uses approximately 5.88 MiB:
- minisatip defaults to approximately 5.51 MiB:
I have limited experience with DVB hardware and production workloads. Do you have a recommended limit based on known hardware or application requirements? I am happy to use the value you recommend in v3.
Quoted 2 lines
Can you provide me more details about the test scenario: e.g. with what TV standards had you test it, and such.
The reproducer used the vidtv virtual DVB adapter (dvb-vidtv-bridge) in a QEMU VM with approximately 3 GiB of RAM. The root-only setup loaded the driver, made dvr0 accessible and set oom_score_adj=-1000 for the process lineage to make the panic deterministic. The ioctl itself was issued by an unprivileged process.
The test did not exercise a particular TV standard or transport stream, since the failure occurs directly in the buffer-allocation ioctl path before stream processing. I have not tested this on physical DVB hardware.
I can share the reproducer package privately if that would be useful.
Best regards,
Cen