Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
aws-us-west-2-korg-lkml-1.web.codeaurora.org
X-Spam-Level:
X-Spam-Status: No, score=1.0 required=3.0 tests=DKIM_ADSP_CUSTOM_MED,
DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,
HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SPF_HELO_NONE,
SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0
Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
by smtp.lore.kernel.org (Postfix) with ESMTP id 1067EC43331
for ; Fri, 27 Mar 2020 19:17:05 +0000 (UTC)
Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mail.kernel.org (Postfix) with ESMTPS id CF35C206E6
for ; Fri, 27 Mar 2020 19:17:04 +0000 (UTC)
Authentication-Results: mail.kernel.org;
dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OH4DvcSr"
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF35C206E6
Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com
Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linux-kernel-mentees-bounces@lists.linuxfoundation.org
Received: from localhost (localhost [127.0.0.1])
by hemlock.osuosl.org (Postfix) with ESMTP id A01DE87E93;
Fri, 27 Mar 2020 19:17:04 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from hemlock.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id 0pAOnLKXmnap; Fri, 27 Mar 2020 19:17:03 +0000 (UTC)
Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56])
by hemlock.osuosl.org (Postfix) with ESMTP id 83D2F87E0B;
Fri, 27 Mar 2020 19:17:03 +0000 (UTC)
Received: from lf-lists.osuosl.org (localhost [127.0.0.1])
by lists.linuxfoundation.org (Postfix) with ESMTP id 65DCCC1D7C;
Fri, 27 Mar 2020 19:17:03 +0000 (UTC)
Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138])
by lists.linuxfoundation.org (Postfix) with ESMTP id D88BCC0177
for ;
Fri, 27 Mar 2020 19:17:01 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by whitealder.osuosl.org (Postfix) with ESMTP id BE389886FD
for ;
Fri, 27 Mar 2020 19:17:01 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
Received: from whitealder.osuosl.org ([127.0.0.1])
by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id FRrDwGtl0ohw
for ;
Fri, 27 Mar 2020 19:17:00 +0000 (UTC)
X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6
Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com
[209.85.222.177])
by whitealder.osuosl.org (Postfix) with ESMTPS id 65221886FA
for ;
Fri, 27 Mar 2020 19:17:00 +0000 (UTC)
Received: by mail-qk1-f177.google.com with SMTP id l25so12069642qki.7
for ;
Fri, 27 Mar 2020 12:17:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=subject:to:cc:references:from:message-id:date:user-agent
:mime-version:in-reply-to:content-language;
bh=An011IuxKqr4K52vHg7C+Gj7D+SUTbrzyKJ+oxWoM2o=;
b=OH4DvcSrG7nljqCiy1DiF9Echt+BZ4b7HAHwgZ9eR65Q3UH98M68xzt9f5SPiNiQx7
Z7Rmim3dqoaxXM2a+Wd1IC5/AWx6Njdw0FUcp+XA7JE8YC+d569AvhGlazuqCyER16JY
74BypznGH7Zn/OS6gAWYv0PfinNTSZTZu28sbj2pec0f7fySm+7uIKZR8y84etuKDPcv
MXQ6PeHHxKZ0LSYsPwRasQtTIiYbriW1T6HUJqy3o5G68CYY5zJmRubLU1u+Xqa8Mnuv
WjfPYoSqRnWkNBFMEf3065xX88yqgjr+KHuMoezrzI0a/jGR5FUyGeAiHCFGecan+64o
1hOA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:subject:to:cc:references:from:message-id:date
:user-agent:mime-version:in-reply-to:content-language;
bh=An011IuxKqr4K52vHg7C+Gj7D+SUTbrzyKJ+oxWoM2o=;
b=WhtDpvvM2xHDDzDq7X9C+300OuIwW5htL4tGhN5PVNyXI1l6hj4lpD+hZCygvn8R75
P/JyswuqatPaekKvosRh+g61ExkMcS65WlXiLZ6KqdKwypis249Lp/9tKd6dmubxX1eA
x/nzakDo9dShDpStOTU46ron/XBitd15LFosRKIYqXxx6PB2zvppU1LLZeYQbK1o4aTX
HoQ2i7HBjun9fxU4zrB26JmoywPR/GK9E9ceCOWK6swITe9IFkomHRo2WaM/pTW3kQmf
wtxbjMw0pZ/mHduUGT6WjWPVMmzGBF696LVcOwd1pzHnk2S+ZCttM3aJIUQerW9Qg3xy
rpEg==
X-Gm-Message-State: ANhLgQ1AMy1ENpo2En9NI6+hQbQLHygK7Lc93cVJoI0RaMAnS9WxlQTw
TBM6VxrGVwcELwaWA7dkQtodHwq7
X-Google-Smtp-Source: ADFU+vunX1gRaaFeLJVgUMbQC8+UYcskH2BmxE8uMFDvPbAg3uQGPBQeJinCm81YGV7pN/r8ATu4cw==
X-Received: by 2002:a37:6455:: with SMTP id y82mr913551qkb.286.1585336618961;
Fri, 27 Mar 2020 12:16:58 -0700 (PDT)
Received: from ?IPv6:2804:14d:72b1:8920:a2ce:f815:f14d:bfac?
([2804:14d:72b1:8920:a2ce:f815:f14d:bfac])
by smtp.gmail.com with ESMTPSA id 85sm4307810qke.128.2020.03.27.12.16.54
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Fri, 27 Mar 2020 12:16:58 -0700 (PDT)
To: Mauro Carvalho Chehab
References: <20200323125732.281976-1-dwlsalmeida@gmail.com>
<20200323125732.281976-4-dwlsalmeida@gmail.com>
<20200327174740.5d5935ae@coco.lan>
From: "Daniel W. S. Almeida"
Message-ID:
Date: Fri, 27 Mar 2020 16:16:50 -0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
Thunderbird/68.6.0
MIME-Version: 1.0
In-Reply-To: <20200327174740.5d5935ae@coco.lan>
Content-Language: en-US
Cc: kstewart@linuxfoundation.org, sean@mess.org, tglx@linutronix.de,
linux-kernel-mentees@lists.linuxfoundation.org, allison@lohutok.net,
linux-media@vger.kernel.org
Subject: Re: [Linux-kernel-mentees] [RFC, WIP,
v2 3/3] media: dvb_dummy_fe.c: write PSI information into DMX buffer
X-BeenThere: linux-kernel-mentees@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id:
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: multipart/mixed; boundary="===============2515832902823073201=="
Errors-To: linux-kernel-mentees-bounces@lists.linuxfoundation.org
Sender: "Linux-kernel-mentees"
This is a multi-part message in MIME format.
--===============2515832902823073201==
Content-Type: multipart/alternative;
boundary="------------5881F56EFBCE801B7772EB35"
Content-Language: en-US
This is a multi-part message in MIME format.
--------------5881F56EFBCE801B7772EB35
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Hi Mauro, as always, thank you for reviewing my code!
Sorry for making you repeat yourself on the alignment of arguments and
on multi-line comment syntax, I am aware of these and I thought I had
fixed them all, but some just slip by sometimes.
> As we talked via IRC in priv, the best would be to implement the MPEG_TS
> generator as part of the bridge DVB driver.
>
> Anyway, I will review the code below assuming that you'll be moving the
> implementation to the right place.
Yes. Please let me know when the changes in experimental/media-kconfig
land, since I'd like to rename and move all the code - including the
bridge I have been working on - to test_drivers/vidtv.
> +static void dvb_dummy_fe_thread_mpeg_ts_tick(struct dvb_frontend *fe)
> +{
> + struct dvb_dummy_fe_state *state = fe->demodulator_priv;
> + const unsigned int SLEEP_MSECS = 10;
> + u32 ticks = 0;
> + u32 i;
> + char *buf = kzalloc(DMX_BUF_LEN, GFP_KERNEL);
> + u32 buffer_offset;
> +
> + struct dvb_dummy_table_pat pat = {0};
> + struct dvb_dummy_table_sdt sdt = {0};
> I guess it is ok here, but allocating too much stuff at the stack is
> dangerous. Linux Kernel stack is very small. Perhaps the best would
> be to place those at the driver's private struct (with is allocated with
> kalloc).
>
Well, I wasn't aware of that, but most of the data for these tables are
heap-allocated. How small are we talking about?
But your suggestion is OK with me as well. It would be even better if
this entire function wasn't in this patch at all, since it will have to
be moved to the bridge driver anyways.
The *_write_args structures are also pretty small.
>> +
>> + struct dvb_dummy_table_pmt *pmt_sections;
>> +
>> + struct dvb_dummy_table_pat_program *programs = NULL;
>> + struct dvb_dummy_table_sdt_service *services = NULL;
>> +
>> + bool update_version_num = false;
>> + u16 pmt_pid;
>> +
>> + programs = dummy_fe_pat_prog_cat_into_new(state->channels);
>> + services = dummy_fe_sdt_serv_cat_into_new(state->channels);
>> +
>> + /* assemble all programs and assign to PAT */
>> + dummy_fe_pat_program_assign(&pat, programs);
>> +
>> + /* assemble all services and assign to SDT */
>> + dummy_fe_sdt_service_assign(&sdt, services);
>> +
>> + /* a section for each program_id */
>> + pmt_sections = kcalloc(pat.programs,
>> + sizeof(struct dvb_dummy_table_pmt),
>> + GFP_KERNEL);
>> +
>> + dummy_fe_pmt_create_section_for_each_pat_entry(&pat,
>> + pmt_sections);
>> +
>> + dummy_fe_pmt_stream_match_with_sections(state->channels,
>> + pmt_sections,
>> + pat.programs);
>> +
>> + dummy_fe_pat_table_init(&pat,
>> + update_version_num,
>> + TRANSPORT_STREAM_ID);
>> +
>> + dummy_fe_sdt_table_init(&sdt,
>> + update_version_num,
>> + TRANSPORT_STREAM_ID);
>> + while (true) {
>> + memset(buf, 0, DMX_BUF_LEN);
>> + buffer_offset = 0;
>> +
>> + if ((ticks % 50) == 0) {
>> + /* push PSI packets into the buffer */
>> +
>> + buffer_offset +=
>> + dummy_fe_pat_write_into(buf,
>> + buffer_offset,
>> + &pat);
>> + for (i = 0; i < pat.programs; ++i) {
>> + pmt_pid =
>> + dummy_fe_pmt_get_pid(&pmt_sections[i],
>> + &pat);
>> +
>> + /* not found */
>> + WARN_ON(pmt_pid > LAST_VALID_TS_PID);
>> +
>> + /* write each section into buffer */
>> + buffer_offset +=
>> + dummy_fe_pmt_write_into(buf,
>> + buffer_offset,
>> + &pmt_sections[i],
>> + pmt_pid);
>> + }
>> +
>> + buffer_offset +=
>> + dummy_fe_sdt_write_into(buf,
>> + buffer_offset,
>> + &sdt);
>> +
>> + WARN_ON(buffer_offset > DMX_BUF_LEN); /* overflow */
>> + msleep_interruptible(SLEEP_MSECS);
> That doesn't sound right, for two reasons:
>
> 1) msleep_interruptible() can take less time than expected, if
> interupted;
> 2) the time may vary a lot.
>
> I would use the high-res timer here, giving a range for it (maybe a 10ms
> range?), e. g., something like:
>
> usleep_range(SLEEP_USECS, SLEEP_USECS + 10000);
>
>
>
OK. I wonder if this will have to be rewritten in the future, since
decoders will probably expect X amount of video/audio per Y amount of time?
As for buffer overflows, maybe a better strategy would be to use a
dynamic array? I would wrap all memcpy() calls and call krealloc() as
necessary. If we go with a big enough initial size (say, 20 TS packets)
then this wouldn't happen very often, if at all. That would be a simple
solution to completely eliminate this problem, in my opinion.
I don't know if there's a limit on how much data you can pass to the
demux at once, but if there is, we can just split the buffer into
smaller chunks when calling dmx_swfilter_packets(), since the amount of
bytes actually present in the buffer will be a multiple of 188 anyways.
> + }
> +
> + length += CRC_SIZE_IN_BYTES;
> +
> + WARN_ON(length > SDT_MAX_SECTION_LEN);
> even assuming that you fix the above code, and update "s" to the next
> SDT data, this is still too dangerous: if are there any risk of going
> past the buffer size, you should check*before* the bad condition happens,
> e. g., something like:
>
> while (s && length + CRC_SIZE_IN_BYTES < SDT_MAX_SECTION_LEN) {
> ...
> }
>
> if (s)
> WARN_ON(length > SDT_MAX_SECTION_LEN);
>
>> + return length;
>> +}
>> +
My understanding is that, e.g.
length > SDT_MAX_SECTION_LEN
doesn't mean that the buffer will necessarily overflow. It is just
against the spec.
Best regards
- Daniel.
--------------5881F56EFBCE801B7772EB35
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi Mauro, as always, thank you for reviewing my code!
Sorry for making you repeat yourself on the alignment of
arguments and on multi-line comment syntax, I am aware of these
and I thought I had fixed them all, but some just slip by
sometimes.
As we talked via IRC in priv, the best would be to implement the MPEG_TS
generator as part of the bridge DVB driver.
Anyway, I will review the code below assuming that you'll be moving the
implementation to the right place.
Yes. Please let me know when the changes in
experimental/media-kconfig land, since I'd like to rename and move
all the code - including the bridge I have been working on - to
test_drivers/vidtv.
+static void dvb_dummy_fe_thread_mpeg_ts_tick(struct dvb_frontend *fe)
+{
+ struct dvb_dummy_fe_state *state = fe->demodulator_priv;
+ const unsigned int SLEEP_MSECS = 10;
+ u32 ticks = 0;
+ u32 i;
+ char *buf = kzalloc(DMX_BUF_LEN, GFP_KERNEL);
+ u32 buffer_offset;
+
+ struct dvb_dummy_table_pat pat = {0};
+ struct dvb_dummy_table_sdt sdt = {0};
I guess it is ok here, but allocating too much stuff at the stack is
dangerous. Linux Kernel stack is very small. Perhaps the best would
be to place those at the driver's private struct (with is allocated with
kalloc).
Well, I wasn't aware of that, but most of the data for these
tables are heap-allocated. How small are we talking about?
But your suggestion is OK with me as well. It would be even
better if this entire function wasn't in this patch at all, since
it will have to be moved to the bridge driver anyways.
The *_write_args structures are also pretty small.
+
+ struct dvb_dummy_table_pmt *pmt_sections;
+
+ struct dvb_dummy_table_pat_program *programs = NULL;
+ struct dvb_dummy_table_sdt_service *services = NULL;
+
+ bool update_version_num = false;
+ u16 pmt_pid;
+
+ programs = dummy_fe_pat_prog_cat_into_new(state->channels);
+ services = dummy_fe_sdt_serv_cat_into_new(state->channels);
+
+ /* assemble all programs and assign to PAT */
+ dummy_fe_pat_program_assign(&pat, programs);
+
+ /* assemble all services and assign to SDT */
+ dummy_fe_sdt_service_assign(&sdt, services);
+
+ /* a section for each program_id */
+ pmt_sections = kcalloc(pat.programs,
+ sizeof(struct dvb_dummy_table_pmt),
+ GFP_KERNEL);
+
+ dummy_fe_pmt_create_section_for_each_pat_entry(&pat,
+ pmt_sections);
+
+ dummy_fe_pmt_stream_match_with_sections(state->channels,
+ pmt_sections,
+ pat.programs);
+
+ dummy_fe_pat_table_init(&pat,
+ update_version_num,
+ TRANSPORT_STREAM_ID);
+
+ dummy_fe_sdt_table_init(&sdt,
+ update_version_num,
+ TRANSPORT_STREAM_ID);
+ while (true) {
+ memset(buf, 0, DMX_BUF_LEN);
+ buffer_offset = 0;
+
+ if ((ticks % 50) == 0) {
+ /* push PSI packets into the buffer */
+
+ buffer_offset +=
+ dummy_fe_pat_write_into(buf,
+ buffer_offset,
+ &pat);
+ for (i = 0; i < pat.programs; ++i) {
+ pmt_pid =
+ dummy_fe_pmt_get_pid(&pmt_sections[i],
+ &pat);
+
+ /* not found */
+ WARN_ON(pmt_pid > LAST_VALID_TS_PID);
+
+ /* write each section into buffer */
+ buffer_offset +=
+ dummy_fe_pmt_write_into(buf,
+ buffer_offset,
+ &pmt_sections[i],
+ pmt_pid);
+ }
+
+ buffer_offset +=
+ dummy_fe_sdt_write_into(buf,
+ buffer_offset,
+ &sdt);
+
+ WARN_ON(buffer_offset > DMX_BUF_LEN); /* overflow */
+ msleep_interruptible(SLEEP_MSECS);
That doesn't sound right, for two reasons:
1) msleep_interruptible() can take less time than expected, if
interupted;
2) the time may vary a lot.
I would use the high-res timer here, giving a range for it (maybe a 10ms
range?), e. g., something like:
usleep_range(SLEEP_USECS, SLEEP_USECS + 10000);
OK. I wonder if this will have to be rewritten in the future,
since decoders will probably expect X amount of video/audio per Y
amount of time?
As for buffer overflows, maybe a better strategy would be to use
a dynamic array? I would wrap all memcpy() calls and call
krealloc() as necessary. If we go with a big enough initial size
(say, 20 TS packets) then this wouldn't happen very often, if at
all. That would be a simple solution to completely eliminate this
problem, in my opinion.
I don't know if there's a limit on how much data you can pass to
the demux at once, but if there is, we can just split the buffer
into smaller chunks when calling dmx_swfilter_packets(), since the
amount of bytes actually present in the buffer will be a multiple
of 188 anyways.
+ }
+
+ length += CRC_SIZE_IN_BYTES;
+
+ WARN_ON(length > SDT_MAX_SECTION_LEN);
even assuming that you fix the above code, and update "s" to the next
SDT data, this is still too dangerous: if are there any risk of going
past the buffer size, you should check *before* the bad condition happens,
e. g., something like:
while (s && length + CRC_SIZE_IN_BYTES < SDT_MAX_SECTION_LEN) {
...
}
if (s)
WARN_ON(length > SDT_MAX_SECTION_LEN);
+ return length;
+}
+
My understanding is that, e.g.
length > SDT_MAX_SECTION_LEN
doesn't mean that the buffer will necessarily overflow. It is
just against the spec.
Best regards
- Daniel.
--------------5881F56EFBCE801B7772EB35--
--===============2515832902823073201==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
--===============2515832902823073201==--