v4l: ti-vpe: Add support for scaling and color conversion
cover v4l: ti-vpe: Add support for scaling and color conversion
The VPE and VIP IPs in DRA7x contain common scaler and color conversion hardware blocks. We create libraries for these components such that the vpe driver and the vip driver(in future) can use these library funcs to configure the blocks.
There are some points for which I would like comments:
- For VPE, setting the format and colorspace for the source and destination queues is enough to determine how these blocks need to be configured and whether they need to be bypassed or not. So it didn't make sense to represent them as media controller entities. For VIP(driver not upstream yet), it's possible that there are multiple data paths which may or may not include these blocks. However, the current use cases don't require such flexibility. There may be a need to re-consider a media controller like setup once we work on the VIP driver. Is it a good idea in terms of user-space compatibilty if we use media controller framework in the future.
- These blocks will also require some custom control commands later on. For example, we may want to tell the scaler later on to perform bi-linear scaling, or perform peaking at a particular frequency.
- The current series keeps the default scaler coefficients in a header file. These coefficients add a lot of lines of code in the kernel. Does it make more sense for the user application to pass the co-efficients to the kernel using an ioctl? Is there any driver which currenlty does this?
The series is based on the branch:
git://linuxtv.org/media_tree.git master
Archit Taneja (8):
v4l: ti-vpe: create a scaler block library
v4l: ti-vpe: support loading of scaler coefficients
v4l: ti-vpe: make vpe driver load scaler coefficients
v4l: ti-vpe: enable basic scaler support
v4l: ti-vpe: create a color space converter block library
v4l: ti-vpe: Add helper to perform color conversion
v4l: ti-vpe: enable CSC support for VPE
v4l: ti-vpe: Add a type specifier to describe vpdma data format type
drivers/media/platform/ti-vpe/Makefile | 2 +- drivers/media/platform/ti-vpe/csc.c | 196 +++++ drivers/media/platform/ti-vpe/csc.h | 68 ++ drivers/media/platform/ti-vpe/sc.c | 311 +++++++ drivers/media/platform/ti-vpe/sc.h | 208 +++++ drivers/media/platform/ti-vpe/sc_coeff.h | 1342 ++++++++++++++++++++++++++++++ drivers/media/platform/ti-vpe/vpdma.c | 36 +- drivers/media/platform/ti-vpe/vpdma.h | 7 + drivers/media/platform/ti-vpe/vpe.c | 251 ++++-- drivers/media/platform/ti-vpe/vpe_regs.h | 187 ----- 10 files changed, 2335 insertions(+), 273 deletions(-) create mode 100644 drivers/media/platform/ti-vpe/csc.c create mode 100644 drivers/media/platform/ti-vpe/csc.h create mode 100644 drivers/media/platform/ti-vpe/sc.c create mode 100644 drivers/media/platform/ti-vpe/sc.h create mode 100644 drivers/media/platform/ti-vpe/sc_coeff.h
Signature · 1 line
1.8.3.2
PATCH 1/8 v4l: ti-vpe: create a scaler block library
VPE and VIP IPs in DAR7x contain a scaler(SC) sub block. Create a library which will perform scaler block related configurations and hold SC register definitions. The functions provided by this library will be called by the vpe and vip drivers using a sc_data handle.
The vpe_dev holds the sc_data handle. The handle represents an instance of the SC hardware, and the vpe driver uses it to access the scaler register offsets or helper functions to configure these registers.
We move the SC register definitions to sc.h so that they aren't specific to VPE anymore. The register offsets are now relative to the sub-block, and not the VPE IP as a whole. In order for VPDMA to configure registers, it requires it's offset from the top level VPE module. A macro called GET_OFFSET_TOP is added to return the offset of the register relative to the VPE IP.
--- drivers/media/platform/ti-vpe/Makefile | 2 +- drivers/media/platform/ti-vpe/sc.c | 91 ++++++++++++++++ drivers/media/platform/ti-vpe/sc.h | 175 +++++++++++++++++++++++++++++++ drivers/media/platform/ti-vpe/vpe.c | 60 ++++------- drivers/media/platform/ti-vpe/vpe_regs.h | 149 -------------------------- 5 files changed, 288 insertions(+), 189 deletions(-) create mode 100644 drivers/media/platform/ti-vpe/sc.c create mode 100644 drivers/media/platform/ti-vpe/sc.h
drivers/media/platform/ti-vpe/Makefile +1 -1
@@ -1,5 +1,5 @@ obj-$(CONFIG_VIDEO_TI_VPE) += ti-vpe.o -ti-vpe-y := vpe.o vpdma.o +ti-vpe-y := vpe.o sc.o vpdma.o ccflags-$(CONFIG_VIDEO_TI_VPE_DEBUG) += -DDEBUG
drivers/media/platform/ti-vpe/sc.c new file +91 -0
91 lines not on this page · raw message
drivers/media/platform/ti-vpe/sc.h new file +175 -0
175 lines not on this page · raw message
drivers/media/platform/ti-vpe/vpe.c +21 -39
141 lines not on this page · raw message
drivers/media/platform/ti-vpe/vpe_regs.h +0 -149
155 lines not on this page · raw message
Signature · 1 line
1.8.3.2
PATCH 2/8 v4l: ti-vpe: support loading of scaler coefficients
In reply to Archit TanejaThe SC block in VPE/VIP contains a SRAM within it. This internal memory requires to be loaded with appropriate scaler coefficients from a contiguous block of memory through VPDMA.
The horizontal and vertical scaler each require 2 sets of scaler coefficients for luma and chroma scaling. The horizontal polyphase scaler requires coefficients for a 32 phase and 8 tap filter. Similarly, the vertical scaler requires coefficients for a 5 tap filter.
The choice of the scaler coefficients depends on the scaling ratio. Add coefficient tables for different scaling ratios in sc_coeffs.h. In the case of horizontal downscaling, we need to consider the change in ratio caused by decimation performed by the horizontal scaler.
In order to load the scaler coefficients via VPDMA, a configuration descriptor is used in block mode. The payload for the descriptor is the scaler coefficients copied to memory. Coefficients for each phase have to be placed in memory in a particular order understood by the scaler hardware.
The choice of the scaler coefficients, and the loading of the coefficients from our tables to a contiguous buffer is managed by the functions sc_set_hs_coefficients and sc_set_vs_coefficients.
The sc_data handle is now added with some parameters to describe the state of the coefficients loaded in the SC block. 'loaded_coeff_h' and 'loaded_coeff_v' hold the address of the last dma buffer which was used by VPDMA to copy coefficients. This information can be used by a vpe mem-to-mem context to decide whether it should load coefficients or not. 'hs_index' and 'vs_index' provide some optimization by preventing loading of coefficients if the scaling ratio didn't change between 2 contexts. 'load_coeff_h' and 'load_coeff_v' tell the vpe/vip driver whether we need to load the coefficients through VPDMA or not.
--- drivers/media/platform/ti-vpe/sc.c | 98 +++ drivers/media/platform/ti-vpe/sc.h | 31 + drivers/media/platform/ti-vpe/sc_coeff.h | 1342 ++++++++++++++++++++++++++++++ 3 files changed, 1471 insertions(+) create mode 100644 drivers/media/platform/ti-vpe/sc_coeff.h
drivers/media/platform/ti-vpe/sc.c +98 -0
@@ -18,6 +18,7 @@#include <linux/slab.h> #include "sc.h" +#include "sc_coeff.h" void sc_set_regs_bypass (struct sc_data *sc,u32 *sc_reg0) {
@@ -61,6 +62,103 @@ void sc_dump_regs(struct sc_data *sc)#undef DUMPREG } +/* +* set the horizontal scaler coefficients according to the ratio of output to +* input widths, after accounting for up to two levels of decimation +*/ +void sc_set_hs_coeffs (struct sc_data *sc,void *addr,unsigned int src_w, +unsigned int dst_w) +{ +int sixteenths; +int idx; +int i, j; +u16 *coeff_h = addr; +const u16 *cp; + +if (dst_w > src_w) { + idx =HS_UP_SCALE ; + }else { +if ((dst_w <<1 ) < src_w) + dst_w <<=1 ;/* first level decimation */ +if ((dst_w <<1 ) < src_w) + dst_w <<=1 ;/* second level decimation */ + +if (dst_w == src_w) { + idx =HS_LE_16_16_SCALE ; + }else { + sixteenths = (dst_w <<4 ) / src_w; +if (sixteenths <8 ) + sixteenths =8 ; + idx =HS_LT_9_16_SCALE + sixteenths -8 ; + } + } + +if (idx == sc->hs_index) +return ; + + cp = scaler_hs_coeffs[idx]; + +for (i =0 ; i <SC_NUM_PHASES *2 ; i++) { +for (j =0 ; j <SC_H_NUM_TAPS ; j++) + *coeff_h++ = *cp++; +/* +* for each phase, the scaler expects space for 8 coefficients +* in it's memory. For the horizontal scaler, we copy the first +* 7 coefficients and skip the last slot to move to the next +* row to hold coefficients for the next phase +*/ + coeff_h +=SC_NUM_TAPS_MEM_ALIGN -SC_H_NUM_TAPS ; + } + + sc->hs_index = idx; + + sc->load_coeff_h = true; +} + +/* +* set the vertical scaler coefficients according to the ratio of output to +* input heights +*/ +void sc_set_vs_coeffs (struct sc_data *sc,void *addr,unsigned int src_h, +unsigned int dst_h) +{ +int sixteenths; +int idx; +int i, j; +u16 *coeff_v = addr; +const u16 *cp; + +if (dst_h > src_h) { + idx =VS_UP_SCALE ; + }else if (dst_h == src_h) { + idx =VS_1_TO_1_SCALE ; + }else { + sixteenths = (dst_h <<4 ) / src_h; +if (sixteenths <8 ) + sixteenths =8 ; + idx =VS_LT_9_16_SCALE + sixteenths -8 ; + } + +if (idx == sc->vs_index) +return ; + + cp = scaler_vs_coeffs[idx]; + +for (i =0 ; i <SC_NUM_PHASES *2 ; i++) { +for (j =0 ; j <SC_V_NUM_TAPS ; j++) + *coeff_v++ = *cp++; +/* +* for the vertical scaler, we copy the first 5 coefficients and +* skip the last 3 slots to move to the next row to hold +* coefficients for the next phase +*/ + coeff_v +=SC_NUM_TAPS_MEM_ALIGN -SC_V_NUM_TAPS ; + } + + sc->vs_index = idx; + sc->load_coeff_v = true; +} +struct sc_data *sc_create (struct platform_device *pdev) {struct sc_data *sc;
drivers/media/platform/ti-vpe/sc.h +31 -0
@@ -161,15 +161,46 @@#define CFG_OFF_W_MASK 0x07ff#define CFG_OFF_W_SHIFT 16 +/* number of phases supported by the polyphase scalers */ +#define SC_NUM_PHASES 32 + +/* number of taps used by horizontal polyphase scaler */ +#define SC_H_NUM_TAPS 7 + +/* number of taps used by vertical polyphase scaler */ +#define SC_V_NUM_TAPS 5 + +/* number of taps expected by the scaler in it's coefficient memory */ +#define SC_NUM_TAPS_MEM_ALIGN 8 + +/* +* coefficient memory size in bytes: +* num phases x num sets(luma and chroma) x num taps(aligned) x coeff size +*/ +#define SC_COEF_SRAM_SIZE (SC_NUM_PHASES * 2 * SC_NUM_TAPS_MEM_ALIGN * 2) +struct sc_data {void __iomem *base;struct resource *res; +dma_addr_t loaded_coeff_h;/* loaded h coeffs in SC */ +dma_addr_t loaded_coeff_v;/* loaded v coeffs in SC */ + +bool load_coeff_h;/* have new h SC coeffs */ +bool load_coeff_v;/* have new v SC coeffs */ + +unsigned int hs_index;/* h SC coeffs selector */ +unsigned int vs_index;/* v SC coeffs selector */ +struct platform_device *pdev; };void sc_set_regs_bypass (struct sc_data *sc,u32 *sc_reg0);void sc_dump_regs (struct sc_data *sc); +void sc_set_hs_coeffs (struct sc_data *sc,void *addr,unsigned int src_w, +unsigned int dst_w); +void sc_set_vs_coeffs (struct sc_data *sc,void *addr,unsigned int src_h, +unsigned int dst_h);struct sc_data *sc_create (struct platform_device *pdev);#endif
drivers/media/platform/ti-vpe/sc_coeff.h new file +1342 -0
1,342 lines not on this page · raw message
Signature · 1 line
1.8.3.2
PATCH 3/8 v4l: ti-vpe: make vpe driver load scaler coefficients
In reply to Archit TanejaMake the driver allocate dma buffers to store horizontal and scaler coeffs. Use the scaler library api to choose and copy scaler coefficients to a the above buffers based on the scaling ratio. Since the SC block comes after the de-interlacer, make sure that the source height is doubled if de-interlacer was used.
These buffers now need to be used by VPDMA to load the coefficients into the SRAM within SC.
In device_run, add configuration descriptors which have payloads pointing to the scaler coefficients in memory. Use the members in sc_data handle to prevent addition of these descriptors if there isn't a need to re-load coefficients into SC. This comes helps unnecessary re-loading of the coefficients when we switch back and forth between vpe contexts.
--- drivers/media/platform/ti-vpe/vpe.c | 47 ++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-)
drivers/media/platform/ti-vpe/vpe.c +46 -1
@@ -356,6 +356,8 @@ struct vpe_ctx {void *mv_buf[2 ];/* virtual addrs of motion vector bufs */ size_t mv_buf_size;/* current motion vector buffer size */ struct vpdma_buf mmr_adb;/* shadow reg addr/data block */ +struct vpdma_buf sc_coeff_h;/* h coeff buffer */ +struct vpdma_buf sc_coeff_v;/* v coeff buffer */ struct vpdma_desc_list desc_list;/* DMA descriptor list */ bool deinterlacing;/* using de-interlacer */
@@ -765,6 +767,10 @@ static int set_srcdst_params(struct vpe_ctx *ctx)struct vpe_q_data *s_q_data = &ctx->q_data[Q_DATA_SRC ];struct vpe_q_data *d_q_data = &ctx->q_data[Q_DATA_DST ];struct vpe_mmr_adb *mmr_adb = ctx->mmr_adb.addr; +unsigned int src_w = s_q_data->c_rect.width; +unsigned int src_h = s_q_data->c_rect.height; +unsigned int dst_w = d_q_data->c_rect.width; +unsigned int dst_h = d_q_data->c_rect.height;size_t mv_buf_size;int ret;
@@ -777,7 +783,6 @@ static int set_srcdst_params(struct vpe_ctx *ctx)const struct vpdma_data_format *mv = &vpdma_misc_fmts[VPDMA_DATA_FMT_MV ]; - ctx->deinterlacing =1 ;/* * we make sure that the source image has a 16 byte aligned * stride, we need to do the same for the motion vector buffer
@@ -788,6 +793,9 @@ static int set_srcdst_params(struct vpe_ctx *ctx) bytes_per_line =ALIGN ((s_q_data->width * mv->depth) >>3 ,VPDMA_STRIDE_ALIGN ); mv_buf_size = bytes_per_line * s_q_data->height; + + ctx->deinterlacing =1 ; + src_h <<=1 ; }else { ctx->deinterlacing =0 ; mv_buf_size =0 ;
@@ -802,6 +810,8 @@ static int set_srcdst_params(struct vpe_ctx *ctx)set_cfg_and_line_modes (ctx);set_dei_regs (ctx);set_csc_coeff_bypass (ctx); +sc_set_hs_coeffs (ctx->dev->sc, ctx->sc_coeff_h.addr, src_w, dst_w); +sc_set_vs_coeffs (ctx->dev->sc, ctx->sc_coeff_v.addr, src_h, dst_h);sc_set_regs_bypass (ctx->dev->sc, &mmr_adb->sc_regs[0 ]);return 0 ;
@@ -1035,6 +1045,7 @@ static void disable_irqs(struct vpe_ctx *ctx)static void device_run (void *priv) {struct vpe_ctx *ctx = priv; +struct sc_data *sc = ctx->dev->sc;struct vpe_q_data *d_q_data = &ctx->q_data[Q_DATA_DST ];if (ctx->deinterlacing && ctx->src_vbs[2 ] ==NULL ) {
@@ -1057,6 +1068,26 @@ static void device_run(void *priv) ctx->load_mmrs = false; } +if (sc->loaded_coeff_h != ctx->sc_coeff_h.dma_addr || + sc->load_coeff_h) { +vpdma_map_desc_buf (ctx->dev->vpdma, &ctx->sc_coeff_h); +vpdma_add_cfd_block (&ctx->desc_list,CFD_SC_CLIENT , + &ctx->sc_coeff_h,0 ); + + sc->loaded_coeff_h = ctx->sc_coeff_h.dma_addr; + sc->load_coeff_h = false; + } + +if (sc->loaded_coeff_v != ctx->sc_coeff_v.dma_addr || + sc->load_coeff_v) { +vpdma_map_desc_buf (ctx->dev->vpdma, &ctx->sc_coeff_v); +vpdma_add_cfd_block (&ctx->desc_list,CFD_SC_CLIENT , + &ctx->sc_coeff_v,SC_COEF_SRAM_SIZE >>4 ); + + sc->loaded_coeff_v = ctx->sc_coeff_v.dma_addr; + sc->load_coeff_v = false; + } +/* output data descriptors */ if (ctx->deinterlacing)add_out_dtd (ctx,VPE_PORT_MV_OUT );
@@ -1180,6 +1211,8 @@ static irqreturn_t vpe_irq(int irq_vpe, void *data)vpdma_unmap_desc_buf (dev->vpdma, &ctx->desc_list.buf);vpdma_unmap_desc_buf (dev->vpdma, &ctx->mmr_adb); +vpdma_unmap_desc_buf (dev->vpdma, &ctx->sc_coeff_h); +vpdma_unmap_desc_buf (dev->vpdma, &ctx->sc_coeff_v);vpdma_reset_desc_list (&ctx->desc_list);
@@ -1752,6 +1785,14 @@ static int vpe_open(struct file *file)if (ret !=0 ) goto free_desc_list; + ret =vpdma_alloc_desc_buf (&ctx->sc_coeff_h,SC_COEF_SRAM_SIZE ); +if (ret !=0 ) + goto free_mmr_adb; + + ret =vpdma_alloc_desc_buf (&ctx->sc_coeff_v,SC_COEF_SRAM_SIZE ); +if (ret !=0 ) + goto free_sc_h; +init_adb_hdrs (ctx);v4l2_fh_init (&ctx->fh,video_devdata (file));
@@ -1820,6 +1861,10 @@ static int vpe_open(struct file *file) exit_fh:v4l2_ctrl_handler_free (hdl);v4l2_fh_exit (&ctx->fh); +vpdma_free_desc_buf (&ctx->sc_coeff_v); +free_sc_h: +vpdma_free_desc_buf (&ctx->sc_coeff_h); +free_mmr_adb:vpdma_free_desc_buf (&ctx->mmr_adb); free_desc_list:vpdma_free_desc_list (&ctx->desc_list);
Signature · 1 line
1.8.3.2
PATCH 4/8 v4l: ti-vpe: enable basic scaler support
In reply to Archit TanejaAdd the required SC register configurations which lets us perform linear scaling for the supported range of horizontal and vertical scaling ratios.
The horizontal scaler performs polyphase scaling using it's 8 tap 32 phase filter, decimation is performed when downscaling passes beyond 2x or 4x.
The vertical scaler performs polyphase scaling using it's 5 tap 32 phase filter, it switches to a simpler form of scaling using the running average filter when the downscale ratio is more than 4x.
Many of the SC features like peaking, trimming and non-linear scaling aren't implemented for now. Only the minimal register fields required for basic scaling operation are configured.
The function to configure SC registers takes the sc_data handle, the source and destination widths and heights, and the scaler address data block offsets for the current context so that they can be configured.
--- drivers/media/platform/ti-vpe/sc.c | 132 ++++++++++++++++++++++++++++++++++-- drivers/media/platform/ti-vpe/sc.h | 4 +- drivers/media/platform/ti-vpe/vpe.c | 24 +++++-- 3 files changed, 149 insertions(+), 11 deletions(-)
drivers/media/platform/ti-vpe/sc.c +127 -5
@@ -20,11 +20,6 @@#include "sc.h" #include "sc_coeff.h" -void sc_set_regs_bypass (struct sc_data *sc,u32 *sc_reg0) -{ - *sc_reg0 |=CFG_SC_BYPASS ; -} -void sc_dump_regs (struct sc_data *sc) {struct device *dev = &sc->pdev->dev;
@@ -159,6 +154,133 @@ void sc_set_vs_coeffs(struct sc_data *sc, void *addr, unsigned int src_h, sc->load_coeff_v = true; } +void sc_config_scaler (struct sc_data *sc,u32 *sc_reg0,u32 *sc_reg8, +u32 *sc_reg17,unsigned int src_w,unsigned int src_h, +unsigned int dst_w,unsigned int dst_h) +{ +struct device *dev = &sc->pdev->dev; +u32 val; +int dcm_x, dcm_shift; +bool use_rav; +unsigned long lltmp; +u32 lin_acc_inc, lin_acc_inc_u; +u32 col_acc_offset; +u16 factor =0 ; +int row_acc_init_rav =0 , row_acc_init_rav_b =0 ; +u32 row_acc_inc =0 , row_acc_offset =0 , row_acc_offset_b =0 ; +/* +* location of SC register in payload memory with respect to the first +* register in the mmr address data block +*/ +u32 *sc_reg9 = sc_reg8 +1 ; +u32 *sc_reg12 = sc_reg8 +4 ; +u32 *sc_reg13 = sc_reg8 +5 ; +u32 *sc_reg24 = sc_reg17 +7 ; + + val = sc_reg0[0 ]; + +/* clear all the features(they may get enabled elsewhere later) */ + val &= ~(CFG_SELFGEN_FID |CFG_TRIM |CFG_ENABLE_SIN2_VER_INTP | +CFG_INTERLACE_I |CFG_DCM_4X |CFG_DCM_2X |CFG_AUTO_HS | +CFG_ENABLE_EV |CFG_USE_RAV |CFG_INVT_FID |CFG_SC_BYPASS | +CFG_INTERLACE_O |CFG_Y_PK_EN |CFG_HP_BYPASS |CFG_LINEAR ); + +if (src_w == dst_w && src_h == dst_h) { + val |=CFG_SC_BYPASS ; + sc_reg0[0 ] = val; +return ; + } + +/* we only support linear scaling for now */ + val |=CFG_LINEAR ; + +/* configure horizontal scaler */ + +/* enable 2X or 4X decimation */ + dcm_x = src_w / dst_w; +if (dcm_x >4 ) { + val |=CFG_DCM_4X ; + dcm_shift =2 ; + }else if (dcm_x >2 ) { + val |=CFG_DCM_2X ; + dcm_shift =1 ; + }else { + dcm_shift =0 ; + } + + lltmp = dst_w -1 ; + lin_acc_inc =div64_u64 (((u64)(src_w >> dcm_shift) -1 ) <<24 , lltmp); + lin_acc_inc_u =0 ; + col_acc_offset =0 ; + +dev_dbg (dev,"hs config: src_w = %d, dst_w = %d, decimation = %s, lin_acc_inc = %08x\n" , + src_w, dst_w, dcm_shift ==2 ?"4x" : + (dcm_shift ==1 ?"2x" :"none" ), lin_acc_inc); + +/* configure vertical scaler */ + +/* use RAV for vertical scaler if vertical downscaling is > 4x */ +if (dst_h < (src_h >>2 )) { + use_rav = true; + val |=CFG_USE_RAV ; + }else { + use_rav = false; + } + +if (use_rav) { +/* use RAV */ + factor = (u16) ((dst_h <<10 ) / src_h); + + row_acc_init_rav = factor + ((1 + factor) >>1 ); +if (row_acc_init_rav >=1024 ) + row_acc_init_rav -=1024 ; + + row_acc_init_rav_b = row_acc_init_rav + + (1 + (row_acc_init_rav >>1 )) - + (1024 >>1 ); + +if (row_acc_init_rav_b <0 ) { + row_acc_init_rav_b += row_acc_init_rav; + row_acc_init_rav *=2 ; + } + +dev_dbg (dev,"vs config(RAV): src_h = %d, dst_h = %d, factor = %d, acc_init = %08x, acc_init_b = %08x\n" , + src_h, dst_h, factor, row_acc_init_rav, + row_acc_init_rav_b); + }else { +/* use polyphase */ + row_acc_inc = ((src_h -1 ) <<16 ) / (dst_h -1 ); + row_acc_offset =0 ; + row_acc_offset_b =0 ; + +dev_dbg (dev,"vs config(POLY): src_h = %d, dst_h = %d,row_acc_inc = %08x\n" , + src_h, dst_h, row_acc_inc); + } + + + sc_reg0[0 ] = val; + sc_reg0[1 ] = row_acc_inc; + sc_reg0[2 ] = row_acc_offset; + sc_reg0[3 ] = row_acc_offset_b; + + sc_reg0[4 ] = ((lin_acc_inc_u &CFG_LIN_ACC_INC_U_MASK ) << +CFG_LIN_ACC_INC_U_SHIFT ) | (dst_w <<CFG_TAR_W_SHIFT ) | + (dst_h <<CFG_TAR_H_SHIFT ); + + sc_reg0[5 ] = (src_w <<CFG_SRC_W_SHIFT ) | (src_h <<CFG_SRC_H_SHIFT ); + + sc_reg0[6 ] = (row_acc_init_rav_b <<CFG_ROW_ACC_INIT_RAV_B_SHIFT ) | + (row_acc_init_rav <<CFG_ROW_ACC_INIT_RAV_SHIFT ); + + *sc_reg9 = lin_acc_inc; + + *sc_reg12 = col_acc_offset <<CFG_COL_ACC_OFFSET_SHIFT ; + + *sc_reg13 = factor; + + *sc_reg24 = (src_w <<CFG_ORG_W_SHIFT ) | (src_h <<CFG_ORG_H_SHIFT ); +} +struct sc_data *sc_create (struct platform_device *pdev) {struct sc_data *sc;
drivers/media/platform/ti-vpe/sc.h +3 -1
@@ -195,12 +195,14 @@ struct sc_data { struct platform_device *pdev; }; -void sc_set_regs_bypass (struct sc_data *sc,u32 *sc_reg0);void sc_dump_regs (struct sc_data *sc);void sc_set_hs_coeffs (struct sc_data *sc,void *addr,unsigned int src_w,unsigned int dst_w);void sc_set_vs_coeffs (struct sc_data *sc,void *addr,unsigned int src_h,unsigned int dst_h); +void sc_config_scaler (struct sc_data *sc,u32 *sc_reg0,u32 *sc_reg8, +u32 *sc_reg17,unsigned int src_w,unsigned int src_h, +unsigned int dst_w,unsigned int dst_h);struct sc_data *sc_create (struct platform_device *pdev);#endif
drivers/media/platform/ti-vpe/vpe.c +19 -5
@@ -440,9 +440,15 @@ struct vpe_mmr_adb {u32 us3_regs[8 ];struct vpdma_adb_hdr dei_hdr;u32 dei_regs[8 ]; -struct vpdma_adb_hdr sc_hdr; -u32 sc_regs[1 ]; -u32 sc_pad[3 ]; +struct vpdma_adb_hdr sc_hdr0; +u32 sc_regs0[7 ]; +u32 sc_pad0[1 ]; +struct vpdma_adb_hdr sc_hdr8; +u32 sc_regs8[6 ]; +u32 sc_pad8[2 ]; +struct vpdma_adb_hdr sc_hdr17; +u32 sc_regs17[9 ]; +u32 sc_pad17[3 ];struct vpdma_adb_hdr csc_hdr;u32 csc_regs[6 ];u32 csc_pad[2 ];
@@ -463,8 +469,12 @@ static void init_adb_hdrs(struct vpe_ctx *ctx)VPE_SET_MMR_ADB_HDR (ctx, us2_hdr, us2_regs,VPE_US2_R0 );VPE_SET_MMR_ADB_HDR (ctx, us3_hdr, us3_regs,VPE_US3_R0 );VPE_SET_MMR_ADB_HDR (ctx, dei_hdr, dei_regs,VPE_DEI_FRAME_SIZE ); -VPE_SET_MMR_ADB_HDR (ctx, sc_hdr, sc_regs, +VPE_SET_MMR_ADB_HDR (ctx, sc_hdr0, sc_regs0,GET_OFFSET_TOP (ctx, ctx->dev->sc,CFG_SC0 )); +VPE_SET_MMR_ADB_HDR (ctx, sc_hdr8, sc_regs8, +GET_OFFSET_TOP (ctx, ctx->dev->sc,CFG_SC8 )); +VPE_SET_MMR_ADB_HDR (ctx, sc_hdr17, sc_regs17, +GET_OFFSET_TOP (ctx, ctx->dev->sc,CFG_SC17 ));VPE_SET_MMR_ADB_HDR (ctx, csc_hdr, csc_regs,VPE_CSC_CSC00 ); };
@@ -810,9 +820,13 @@ static int set_srcdst_params(struct vpe_ctx *ctx)set_cfg_and_line_modes (ctx);set_dei_regs (ctx);set_csc_coeff_bypass (ctx); +sc_set_hs_coeffs (ctx->dev->sc, ctx->sc_coeff_h.addr, src_w, dst_w);sc_set_vs_coeffs (ctx->dev->sc, ctx->sc_coeff_v.addr, src_h, dst_h); -sc_set_regs_bypass (ctx->dev->sc, &mmr_adb->sc_regs[0 ]); + +sc_config_scaler (ctx->dev->sc, &mmr_adb->sc_regs0[0 ], + &mmr_adb->sc_regs8[0 ], &mmr_adb->sc_regs17[0 ], + src_w, src_h, dst_w, dst_h);return 0 ; }
Signature · 1 line
1.8.3.2
PATCH 5/8 v4l: ti-vpe: create a color space converter block library
In reply to Archit TanejaVPE and VIP IPs in DAR7x contain a color space converter(CSC) sub block. Create a library which will perform CSC related configurations and hold CSC register definitions. The functions provided by this library will be called by the vpe and vip drivers using a csc_data handle.
The vpe_dev holds the csc_data handle. The handle represents an instance of the CSC hardware, and the vpe driver uses it to access the CSC register offsets or helper functions to configure these registers.
The CSC register offsets are now relative to the CSC block itself, so we need to use the macro GET_OFFSET_TOP to get the CSC register offset relative to the VPE IP in the vpe driver.
--- drivers/media/platform/ti-vpe/Makefile | 2 +- drivers/media/platform/ti-vpe/csc.c | 76 ++++++++++++++++++++++++++++++++ drivers/media/platform/ti-vpe/csc.h | 65 +++++++++++++++++++++++++++ drivers/media/platform/ti-vpe/vpe.c | 31 ++++++------- drivers/media/platform/ti-vpe/vpe_regs.h | 38 ---------------- 5 files changed, 155 insertions(+), 57 deletions(-) create mode 100644 drivers/media/platform/ti-vpe/csc.c create mode 100644 drivers/media/platform/ti-vpe/csc.h
drivers/media/platform/ti-vpe/Makefile +1 -1
@@ -1,5 +1,5 @@ obj-$(CONFIG_VIDEO_TI_VPE) += ti-vpe.o -ti-vpe-y := vpe.o sc.o vpdma.o +ti-vpe-y := vpe.o sc.o csc.o vpdma.o ccflags-$(CONFIG_VIDEO_TI_VPE_DEBUG) += -DDEBUG
drivers/media/platform/ti-vpe/csc.c new file +76 -0
76 lines not on this page · raw message
drivers/media/platform/ti-vpe/csc.h new file +65 -0
65 lines not on this page · raw message
drivers/media/platform/ti-vpe/vpe.c +13 -18
@@ -44,6 +44,7 @@#include "vpdma.h" #include "vpe_regs.h" #include "sc.h" +#include "csc.h" #define VPE_MODULE_NAME "vpe"
@@ -330,6 +331,7 @@ struct vpe_dev {struct vb2_alloc_ctx *alloc_ctx;struct vpdma_data *vpdma;/* vpdma data handle */ struct sc_data *sc;/* scaler data handle */ +struct csc_data *csc;/* csc data handle */ };/*
@@ -475,7 +477,8 @@ static void init_adb_hdrs(struct vpe_ctx *ctx) GET_OFFSET_TOP(ctx, ctx->dev->sc, CFG_SC8));VPE_SET_MMR_ADB_HDR (ctx, sc_hdr17, sc_regs17,GET_OFFSET_TOP (ctx, ctx->dev->sc,CFG_SC17 )); -VPE_SET_MMR_ADB_HDR (ctx, csc_hdr, csc_regs,VPE_CSC_CSC00 ); +VPE_SET_MMR_ADB_HDR (ctx, csc_hdr, csc_regs, +GET_OFFSET_TOP (ctx, ctx->dev->csc,CSC_CSC00 )); };/*
@@ -758,16 +761,6 @@ static void set_dei_shadow_registers(struct vpe_ctx *ctx) ctx->load_mmrs = true; } -static void set_csc_coeff_bypass (struct vpe_ctx *ctx) -{ -struct vpe_mmr_adb *mmr_adb = ctx->mmr_adb.addr; -u32 *shadow_csc_reg5 = &mmr_adb->csc_regs[5 ]; - - *shadow_csc_reg5 |=VPE_CSC_BYPASS ; - - ctx->load_mmrs = true; -} -/* * Set the shadow registers whose values are modified when either the * source or destination format is changed.
@@ -819,7 +812,8 @@ static int set_srcdst_params(struct vpe_ctx *ctx)set_cfg_and_line_modes (ctx);set_dei_regs (ctx); -set_csc_coeff_bypass (ctx); + +csc_set_coeff_bypass (ctx->dev->csc, &mmr_adb->csc_regs[5 ]);sc_set_hs_coeffs (ctx->dev->sc, ctx->sc_coeff_h.addr, src_w, dst_w);sc_set_vs_coeffs (ctx->dev->sc, ctx->sc_coeff_v.addr, src_h, dst_h);
@@ -942,15 +936,10 @@ static void vpe_dump_regs(struct vpe_dev *dev)DUMPREG (DEI_FMD_STATUS_R0 );DUMPREG (DEI_FMD_STATUS_R1 );DUMPREG (DEI_FMD_STATUS_R2 ); -DUMPREG (CSC_CSC00 ); -DUMPREG (CSC_CSC01 ); -DUMPREG (CSC_CSC02 ); -DUMPREG (CSC_CSC03 ); -DUMPREG (CSC_CSC04 ); -DUMPREG (CSC_CSC05 );#undef DUMPREGsc_dump_regs (dev->sc); +csc_dump_regs (dev->csc); }static void add_out_dtd (struct vpe_ctx *ctx,int port)
@@ -2074,6 +2063,12 @@ static int vpe_probe(struct platform_device *pdev) goto runtime_put; } + dev->csc =csc_create (pdev); +if (IS_ERR (dev->csc)) { + ret =PTR_ERR (dev->csc); + goto runtime_put; + } + dev->vpdma =vpdma_create (pdev);if (IS_ERR (dev->vpdma)) { ret =PTR_ERR (dev->vpdma);
drivers/media/platform/ti-vpe/vpe_regs.h +0 -38
@@ -306,42 +306,4 @@#define VPE_FMD_FRAME_DIFF_MASK 0x000fffff#define VPE_FMD_FRAME_DIFF_SHIFT 0 -/* VPE color space converter regs */ -#define VPE_CSC_CSC00 0x5700 -#define VPE_CSC_A0_MASK 0x1fff -#define VPE_CSC_A0_SHIFT 0 -#define VPE_CSC_B0_MASK 0x1fff -#define VPE_CSC_B0_SHIFT 16 - -#define VPE_CSC_CSC01 0x5704 -#define VPE_CSC_C0_MASK 0x1fff -#define VPE_CSC_C0_SHIFT 0 -#define VPE_CSC_A1_MASK 0x1fff -#define VPE_CSC_A1_SHIFT 16 - -#define VPE_CSC_CSC02 0x5708 -#define VPE_CSC_B1_MASK 0x1fff -#define VPE_CSC_B1_SHIFT 0 -#define VPE_CSC_C1_MASK 0x1fff -#define VPE_CSC_C1_SHIFT 16 - -#define VPE_CSC_CSC03 0x570c -#define VPE_CSC_A2_MASK 0x1fff -#define VPE_CSC_A2_SHIFT 0 -#define VPE_CSC_B2_MASK 0x1fff -#define VPE_CSC_B2_SHIFT 16 - -#define VPE_CSC_CSC04 0x5710 -#define VPE_CSC_C2_MASK 0x1fff -#define VPE_CSC_C2_SHIFT 0 -#define VPE_CSC_D0_MASK 0x0fff -#define VPE_CSC_D0_SHIFT 16 - -#define VPE_CSC_CSC05 0x5714 -#define VPE_CSC_D1_MASK 0x0fff -#define VPE_CSC_D1_SHIFT 0 -#define VPE_CSC_D2_MASK 0x0fff -#define VPE_CSC_D2_SHIFT 16 -#define VPE_CSC_BYPASS (1 << 28) -#endif
Signature · 1 line
1.8.3.2
PATCH 6/8 v4l: ti-vpe: Add helper to perform color conversion
In reply to Archit TanejaThe CSC block can be used for color space conversion between YUV and RGB formats.
It is configurable via a programmable set of coefficients. Add functionality to choose the appropriate CSC coefficients and program them in the CSC registers. We take the source and destination colorspace formats as the arguments, and choose the coefficient table accordingly.
YUV to RGB coefficients are provided for standard and high definition colorspaces. The coefficients can also be limited or full range. For now, only full range coefficients are chosen. We would need some sort of control ioctl for the user to specify the range needed. Not sure if there is a generic control ioctl for this already?
--- drivers/media/platform/ti-vpe/csc.c | 120 ++++++++++++++++++++++++++++++++++++ drivers/media/platform/ti-vpe/csc.h | 3 + 2 files changed, 123 insertions(+)
drivers/media/platform/ti-vpe/csc.c +120 -0
@@ -16,9 +16,79 @@#include <linux/io.h> #include <linux/platform_device.h> #include <linux/slab.h> +#include <linux/videodev2.h> #include "csc.h" +/* +* 16 coefficients in the order: +* a0, b0, c0, a1, b1, c1, a2, b2, c2, d0, d1, d2 +* (we may need to pass non-default values from user space later on, we might +* need to make the coefficient struct more easy to populate) +*/ +struct colorspace_coeffs { +u16 sd[12 ]; +u16 hd[12 ]; +}; + +/* VIDEO_RANGE: limited range, GRAPHICS_RANGE: full range */ +#define CSC_COEFFS_VIDEO_RANGE_Y2R 0 +#define CSC_COEFFS_GRAPHICS_RANGE_Y2R 1 +#define CSC_COEFFS_VIDEO_RANGE_R2Y 2 +#define CSC_COEFFS_GRAPHICS_RANGE_R2Y 3 + +/* default colorspace coefficients */ +static struct colorspace_coeffs colorspace_coeffs[4 ] = { + [CSC_COEFFS_VIDEO_RANGE_Y2R ] = { + { +/* SDTV */ +0x0400 ,0x0000 ,0x057D ,0x0400 ,0x1EA7 ,0x1D35 , +0x0400 ,0x06EF ,0x1FFE ,0x0D40 ,0x0210 ,0x0C88 , + }, + { +/* HDTV */ +0x0400 ,0x0000 ,0x0629 ,0x0400 ,0x1F45 ,0x1E2B , +0x0400 ,0x0742 ,0x0000 ,0x0CEC ,0x0148 ,0x0C60 , + }, + }, + [CSC_COEFFS_GRAPHICS_RANGE_Y2R ] = { + { +/* SDTV */ +0x04A8 ,0x1FFE ,0x0662 ,0x04A8 ,0x1E6F ,0x1CBF , +0x04A8 ,0x0812 ,0x1FFF ,0x0C84 ,0x0220 ,0x0BAC , + }, + { +/* HDTV */ +0x04A8 ,0x0000 ,0x072C ,0x04A8 ,0x1F26 ,0x1DDE , +0x04A8 ,0x0873 ,0x0000 ,0x0C20 ,0x0134 ,0x0B7C , + }, + }, + [CSC_COEFFS_VIDEO_RANGE_R2Y ] = { + { +/* SDTV */ +0x0132 ,0x0259 ,0x0075 ,0x1F50 ,0x1EA5 ,0x020B , +0x020B ,0x1E4A ,0x1FAB ,0x0000 ,0x0200 ,0x0200 , + }, + { +/* HDTV */ +0x00DA ,0x02DC ,0x004A ,0x1F88 ,0x1E6C ,0x020C , +0x020C ,0x1E24 ,0x1FD0 ,0x0000 ,0x0200 ,0x0200 , + }, + }, + [CSC_COEFFS_GRAPHICS_RANGE_R2Y ] = { + { +/* SDTV */ +0x0107 ,0x0204 ,0x0064 ,0x1F68 ,0x1ED6 ,0x01C2 , +0x01C2 ,0x1E87 ,0x1FB7 ,0x0040 ,0x0200 ,0x0200 , + }, + { +/* HDTV */ +0x04A8 ,0x0000 ,0x072C ,0x04A8 ,0x1F26 ,0x1DDE , +0x04A8 ,0x0873 ,0x0000 ,0x0C20 ,0x0134 ,0x0B7C , + }, + }, +}; +void csc_dump_regs (struct csc_data *csc) {struct device *dev = &csc->pdev->dev;
@@ -45,6 +115,56 @@ void csc_set_coeff_bypass(struct csc_data *csc, u32 *csc_reg5) *csc_reg5 |=CSC_BYPASS ; } +/* +* set the color space converter coefficient shadow register values +*/ +void csc_set_coeff (struct csc_data *csc,u32 *csc_reg0, +enum v4l2_colorspace src_colorspace, +enum v4l2_colorspace dst_colorspace) +{ +u32 *csc_reg5 = csc_reg0 +5 ; +u32 *shadow_csc = csc_reg0; +struct colorspace_coeffs *sd_hd_coeffs; +u16 *coeff, *end_coeff; +enum v4l2_colorspace yuv_colorspace; +int sel =0 ; + +/* +* support only graphics data range(full range) for now, a control ioctl +* would be nice here +*/ +/* Y2R */ +if (dst_colorspace ==V4L2_COLORSPACE_SRGB && + (src_colorspace ==V4L2_COLORSPACE_SMPTE170M || + src_colorspace ==V4L2_COLORSPACE_REC709 )) { +/* Y2R */ + sel =1 ; + yuv_colorspace = src_colorspace; + }else if ((dst_colorspace ==V4L2_COLORSPACE_SMPTE170M || + dst_colorspace ==V4L2_COLORSPACE_REC709 ) && + src_colorspace ==V4L2_COLORSPACE_SRGB ) { +/* R2Y */ + sel =3 ; + yuv_colorspace = dst_colorspace; + }else { + *csc_reg5 |=CSC_BYPASS ; +return ; + } + + sd_hd_coeffs = &colorspace_coeffs[sel]; + +/* select between SD or HD coefficients */ +if (yuv_colorspace ==V4L2_COLORSPACE_SMPTE170M ) + coeff = sd_hd_coeffs->sd; +else + coeff = sd_hd_coeffs->hd; + + end_coeff = coeff +12 ; + +for (; coeff < end_coeff; coeff +=2 ) + *shadow_csc++ = (*(coeff +1 ) <<16 ) | *coeff; +} +struct csc_data *csc_create (struct platform_device *pdev) {struct csc_data *csc;
drivers/media/platform/ti-vpe/csc.h +3 -0
@@ -60,6 +60,9 @@ struct csc_data {void csc_dump_regs (struct csc_data *csc);void csc_set_coeff_bypass (struct csc_data *csc,u32 *csc_reg5); +void csc_set_coeff (struct csc_data *csc,u32 *csc_reg0, +enum v4l2_colorspace src_colorspace, +enum v4l2_colorspace dst_colorspace);struct csc_data *csc_create (struct platform_device *pdev);#endif
Signature · 1 line
1.8.3.2
PATCH 7/8 v4l: ti-vpe: enable CSC support for VPE
In reply to Archit TanejaUse the csc library functions to configure the CSC block in VPE.
Some changes are required in try_fmt to handle the pix->colorspace parameter more correctly. Previously, we copied the source queue colorspace to the destination queue colorspace as we didn't support RGB formats. Now, we configure pix->colorspace based on the color format set(and the height of the image if it's a YUV format).
Add basic RGB color formats to the list of supported vpe formats.
If the destination format is RGB colorspace, we also need to use the RGB output port instead of the Luma and Chroma output ports. This requires configuring the output data descriptors differently.
Also, make the default colorspace V4L2_COLORSPACE_SMPTE170M as that resembles the Standard Definition colorspace more closely.
--- drivers/media/platform/ti-vpe/vpe.c | 95 ++++++++++++++++++++++++++++--------- 1 file changed, 72 insertions(+), 23 deletions(-)
drivers/media/platform/ti-vpe/vpe.c +72 -23
@@ -267,6 +267,38 @@ static struct vpe_fmt vpe_formats[] = { .vpdma_fmt = { &vpdma_yuv_fmts[VPDMA_DATA_FMT_CY422], }, }, + { + .name = "RGB888 packed", + .fourcc = V4L2_PIX_FMT_RGB24, + .types = VPE_FMT_TYPE_CAPTURE, + .coplanar = 0, + .vpdma_fmt = { &vpdma_rgb_fmts[VPDMA_DATA_FMT_RGB24], + }, + }, + { + .name = "ARGB32", + .fourcc = V4L2_PIX_FMT_RGB32, + .types = VPE_FMT_TYPE_CAPTURE, + .coplanar = 0, + .vpdma_fmt = { &vpdma_rgb_fmts[VPDMA_DATA_FMT_ARGB32], + }, + }, + { + .name = "BGR888 packed", + .fourcc = V4L2_PIX_FMT_BGR24, + .types = VPE_FMT_TYPE_CAPTURE, + .coplanar = 0, + .vpdma_fmt = { &vpdma_rgb_fmts[VPDMA_DATA_FMT_BGR24], + }, + }, + { + .name = "ABGR32", + .fourcc = V4L2_PIX_FMT_BGR32, + .types = VPE_FMT_TYPE_CAPTURE, + .coplanar = 0, + .vpdma_fmt = { &vpdma_rgb_fmts[VPDMA_DATA_FMT_ABGR32], + }, + }, };/*
@@ -689,17 +721,20 @@ static void set_src_registers(struct vpe_ctx *ctx)static void set_dst_registers (struct vpe_ctx *ctx) {struct vpe_mmr_adb *mmr_adb = ctx->mmr_adb.addr; +enum v4l2_colorspace clrspc = ctx->q_data[Q_DATA_DST ].colorspace;struct vpe_fmt *fmt = ctx->q_data[Q_DATA_DST ].fmt;u32 val =0 ; -/* select RGB path when color space conversion is supported in future */ -if (fmt->fourcc ==V4L2_PIX_FMT_RGB24 ) - val |=VPE_RGB_OUT_SELECT |VPE_CSC_SRC_DEI_SCALER ; +if (clrspc ==V4L2_COLORSPACE_SRGB ) + val |=VPE_RGB_OUT_SELECT ;else if (fmt->fourcc ==V4L2_PIX_FMT_NV16 ) val |=VPE_COLOR_SEPARATE_422 ; -/* The source of CHR_DS is always the scaler, whether it's used or not */ - val |=VPE_DS_SRC_DEI_SCALER ; +/* +* the source of CHR_DS and CSC is always the scaler, irrespective of +* whether it's used or not +*/ + val |=VPE_DS_SRC_DEI_SCALER |VPE_CSC_SRC_DEI_SCALER ;if (fmt->fourcc !=V4L2_PIX_FMT_NV12 ) val |=VPE_DS_BYPASS ;
@@ -813,7 +848,8 @@ static int set_srcdst_params(struct vpe_ctx *ctx)set_cfg_and_line_modes (ctx);set_dei_regs (ctx); -csc_set_coeff_bypass (ctx->dev->csc, &mmr_adb->csc_regs[5 ]); +csc_set_coeff (ctx->dev->csc, &mmr_adb->csc_regs[0 ], + s_q_data->colorspace, d_q_data->colorspace);sc_set_hs_coeffs (ctx->dev->sc, ctx->sc_coeff_h.addr, src_w, dst_w);sc_set_vs_coeffs (ctx->dev->sc, ctx->sc_coeff_v.addr, src_h, dst_h);
@@ -1095,9 +1131,13 @@ static void device_run(void *priv)if (ctx->deinterlacing)add_out_dtd (ctx,VPE_PORT_MV_OUT ); -add_out_dtd (ctx,VPE_PORT_LUMA_OUT ); -if (d_q_data->fmt->coplanar) -add_out_dtd (ctx,VPE_PORT_CHROMA_OUT ); +if (d_q_data->colorspace ==V4L2_COLORSPACE_SRGB ) { +add_out_dtd (ctx,VPE_PORT_RGB_OUT ); + }else { +add_out_dtd (ctx,VPE_PORT_LUMA_OUT ); +if (d_q_data->fmt->coplanar) +add_out_dtd (ctx,VPE_PORT_CHROMA_OUT ); + }/* input data descriptors */ if (ctx->deinterlacing) {
@@ -1133,9 +1173,16 @@ static void device_run(void *priv) }/* sync on channel control descriptors for output ports */ -vpdma_add_sync_on_channel_ctd (&ctx->desc_list,VPE_CHAN_LUMA_OUT ); -if (d_q_data->fmt->coplanar) -vpdma_add_sync_on_channel_ctd (&ctx->desc_list,VPE_CHAN_CHROMA_OUT ); +if (d_q_data->colorspace ==V4L2_COLORSPACE_SRGB ) { +vpdma_add_sync_on_channel_ctd (&ctx->desc_list, +VPE_CHAN_RGB_OUT ); + }else { +vpdma_add_sync_on_channel_ctd (&ctx->desc_list, +VPE_CHAN_LUMA_OUT ); +if (d_q_data->fmt->coplanar) +vpdma_add_sync_on_channel_ctd (&ctx->desc_list, +VPE_CHAN_CHROMA_OUT ); + }if (ctx->deinterlacing)vpdma_add_sync_on_channel_ctd (&ctx->desc_list,VPE_CHAN_MV_OUT );
@@ -1413,16 +1460,18 @@ static int __vpe_try_fmt(struct vpe_ctx *ctx, struct v4l2_format *f, pix->num_planes = fmt->coplanar ?2 :1 ; pix->pixelformat = fmt->fourcc; -if (type ==VPE_FMT_TYPE_CAPTURE ) { -struct vpe_q_data *s_q_data; - -/* get colorspace from the source queue */ - s_q_data =get_q_data (ctx,V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE ); - - pix->colorspace = s_q_data->colorspace; - }else { -if (!pix->colorspace) - pix->colorspace =V4L2_COLORSPACE_SMPTE240M ; +if (!pix->colorspace) { +if (fmt->fourcc ==V4L2_PIX_FMT_RGB24 || + fmt->fourcc ==V4L2_PIX_FMT_BGR24 || + fmt->fourcc ==V4L2_PIX_FMT_RGB32 || + fmt->fourcc ==V4L2_PIX_FMT_BGR32 ) { + pix->colorspace =V4L2_COLORSPACE_SRGB ; + }else { +if (pix->height >1280 )/* HD */ + pix->colorspace =V4L2_COLORSPACE_REC709 ; +else /* SD */ + pix->colorspace =V4L2_COLORSPACE_SMPTE170M ; + } }for (i =0 ; i < pix->num_planes; i++) {
@@ -1817,7 +1866,7 @@ static int vpe_open(struct file *file) s_q_data->height =1080 ; s_q_data->sizeimage[VPE_LUMA ] = (s_q_data->width * s_q_data->height * s_q_data->fmt->vpdma_fmt[VPE_LUMA ]->depth) >>3 ; - s_q_data->colorspace =V4L2_COLORSPACE_SMPTE240M ; + s_q_data->colorspace =V4L2_COLORSPACE_SMPTE170M ; s_q_data->field =V4L2_FIELD_NONE ; s_q_data->c_rect.left =0 ; s_q_data->c_rect.top =0 ;
Signature · 1 line
1.8.3.2
PATCH 8/8 v4l: ti-vpe: Add a type specifier to describe vpdma data format type
In reply to Archit TanejaThe struct vpdma_data_format holds the color format depth and the data_type value needed to be programmed in the data descriptors. However, it doesn't tell what type of color format is it, i.e, whether it is RGB, YUV or Misc.
This information is needed when by vpdma library when forming descriptors. We modify the depth parameter for the chroma portion of the NV12 format. For this, we check if the data_type value is C420. This isn't sufficient as there are many YUV and RGB vpdma formats which have the same data_type value. Hence, we need to hold the type of the color format for the above case, and possibly more cases in the future.
--- drivers/media/platform/ti-vpe/vpdma.c | 36 +++++++++++++++++++++++++++++++++-- drivers/media/platform/ti-vpe/vpdma.h | 7 +++++++ 2 files changed, 41 insertions(+), 2 deletions(-)
drivers/media/platform/ti-vpe/vpdma.c +34 -2
@@ -30,38 +30,47 @@const struct vpdma_data_format vpdma_yuv_fmts[] = { [VPDMA_DATA_FMT_Y444 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_Y444 , .depth =8 , }, [VPDMA_DATA_FMT_Y422 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_Y422 , .depth =8 , }, [VPDMA_DATA_FMT_Y420 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_Y420 , .depth =8 , }, [VPDMA_DATA_FMT_C444 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_C444 , .depth =8 , }, [VPDMA_DATA_FMT_C422 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_C422 , .depth =8 , }, [VPDMA_DATA_FMT_C420 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_C420 , .depth =4 , }, [VPDMA_DATA_FMT_YC422 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_YC422 , .depth =16 , }, [VPDMA_DATA_FMT_YC444 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_YC444 , .depth =24 , }, [VPDMA_DATA_FMT_CY422 ] = { + .type =VPDMA_DATA_FMT_TYPE_YUV , .data_type =DATA_TYPE_CY422 , .depth =16 , },
@@ -69,82 +78,102 @@ const struct vpdma_data_format vpdma_yuv_fmts[] = {const struct vpdma_data_format vpdma_rgb_fmts[] = { [VPDMA_DATA_FMT_RGB565 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_RGB16_565 , .depth =16 , }, [VPDMA_DATA_FMT_ARGB16_1555 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_ARGB_1555 , .depth =16 , }, [VPDMA_DATA_FMT_ARGB16 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_ARGB_4444 , .depth =16 , }, [VPDMA_DATA_FMT_RGBA16_5551 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_RGBA_5551 , .depth =16 , }, [VPDMA_DATA_FMT_RGBA16 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_RGBA_4444 , .depth =16 , }, [VPDMA_DATA_FMT_ARGB24 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_ARGB24_6666 , .depth =24 , }, [VPDMA_DATA_FMT_RGB24 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_RGB24_888 , .depth =24 , }, [VPDMA_DATA_FMT_ARGB32 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_ARGB32_8888 , .depth =32 , }, [VPDMA_DATA_FMT_RGBA24 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_RGBA24_6666 , .depth =24 , }, [VPDMA_DATA_FMT_RGBA32 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_RGBA32_8888 , .depth =32 , }, [VPDMA_DATA_FMT_BGR565 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_BGR16_565 , .depth =16 , }, [VPDMA_DATA_FMT_ABGR16_1555 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_ABGR_1555 , .depth =16 , }, [VPDMA_DATA_FMT_ABGR16 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_ABGR_4444 , .depth =16 , }, [VPDMA_DATA_FMT_BGRA16_5551 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_BGRA_5551 , .depth =16 , }, [VPDMA_DATA_FMT_BGRA16 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_BGRA_4444 , .depth =16 , }, [VPDMA_DATA_FMT_ABGR24 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_ABGR24_6666 , .depth =24 , }, [VPDMA_DATA_FMT_BGR24 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_BGR24_888 , .depth =24 , }, [VPDMA_DATA_FMT_ABGR32 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_ABGR32_8888 , .depth =32 , }, [VPDMA_DATA_FMT_BGRA24 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_BGRA24_6666 , .depth =24 , }, [VPDMA_DATA_FMT_BGRA32 ] = { + .type =VPDMA_DATA_FMT_TYPE_RGB , .data_type =DATA_TYPE_BGRA32_8888 , .depth =32 , },
@@ -152,6 +181,7 @@ const struct vpdma_data_format vpdma_rgb_fmts[] = {const struct vpdma_data_format vpdma_misc_fmts[] = { [VPDMA_DATA_FMT_MV ] = { + .type =VPDMA_DATA_FMT_TYPE_MISC , .data_type =DATA_TYPE_MV , .depth =4 , },
@@ -599,7 +629,8 @@ void vpdma_add_out_dtd(struct vpdma_desc_list *list, struct v4l2_rect *c_rect, channel = next_chan = chan_info[chan].num; -if (fmt->data_type ==DATA_TYPE_C420 ) +if (fmt->type ==VPDMA_DATA_FMT_TYPE_YUV && + fmt->data_type ==DATA_TYPE_C420 ) depth =8 ; stride =ALIGN ((depth * c_rect->width) >>3 ,VPDMA_STRIDE_ALIGN );
@@ -649,7 +680,8 @@ void vpdma_add_in_dtd(struct vpdma_desc_list *list, int frame_width, channel = next_chan = chan_info[chan].num; -if (fmt->data_type ==DATA_TYPE_C420 ) { +if (fmt->type ==VPDMA_DATA_FMT_TYPE_YUV && + fmt->data_type ==DATA_TYPE_C420 ) { height >>=1 ; frame_height >>=1 ; depth =8 ;
drivers/media/platform/ti-vpe/vpdma.h +7 -0
@@ -39,7 +39,14 @@ struct vpdma_data {bool ready; }; +enum vpdma_data_format_type { +VPDMA_DATA_FMT_TYPE_YUV , +VPDMA_DATA_FMT_TYPE_RGB , +VPDMA_DATA_FMT_TYPE_MISC , +}; +struct vpdma_data_format { +enum vpdma_data_format_type type;int data_type;u8 depth; };
Signature · 1 line
1.8.3.2
Quoted 6 lines · Archit Taneja
The VPE and VIP IPs in DRA7x contain common scaler and color conversion hardware blocks. We create libraries for these components such that the vpe driver and the vip driver(in future) can use these library funcs to configure the blocks.
There are some points for which I would like comments:
Quoted 9 lines
- For VPE, setting the format and colorspace for the source and destination queues is enough to determine how these blocks need to be configured and whether they need to be bypassed or not. So it didn't make sense to represent them as media controller entities. For VIP(driver not upstream yet), it's possible that there are multiple data paths which may or may not include these blocks. However, the current use cases don't require such flexibility. There may be a need to re-consider a media controller like setup once we work on the VIP driver. Is it a good idea in terms of user-space compatibilty if we use media controller framework in the future.
As long as you don't need the mc, then there is no need to implement it.
Quoted 8 lines
- These blocks will also require some custom control commands later on. For example, we may want to tell the scaler later on to perform bi-linear scaling, or perform peaking at a particular frequency.
- The current series keeps the default scaler coefficients in a header file. These coefficients add a lot of lines of code in the kernel. Does it make more sense for the user application to pass the co-efficients to the kernel using an ioctl? Is there any driver which currenlty does this?
I think it is good to keep it in the driver. Otherwise apps would be forced to set up the table. It's about 11 kilobyte in memory, which isn't that bad.
Quoted 15 lines
The series is based on the branch:
git://linuxtv.org/media_tree.gitmasterArchit Taneja (8):
v4l: ti-vpe: create a scaler block library
v4l: ti-vpe: support loading of scaler coefficients
v4l: ti-vpe: make vpe driver load scaler coefficients
v4l: ti-vpe: enable basic scaler support
v4l: ti-vpe: create a color space converter block library
v4l: ti-vpe: Add helper to perform color conversion
v4l: ti-vpe: enable CSC support for VPE
v4l: ti-vpe: Add a type specifier to describe vpdma data format type
Quoted 17 lines
drivers/media/platform/ti-vpe/Makefile | 2 +- drivers/media/platform/ti-vpe/csc.c | 196 +++++ drivers/media/platform/ti-vpe/csc.h | 68 ++ drivers/media/platform/ti-vpe/sc.c | 311 +++++++ drivers/media/platform/ti-vpe/sc.h | 208 +++++ drivers/media/platform/ti-vpe/sc_coeff.h | 1342 ++++++++++++++++++++++++++++++ drivers/media/platform/ti-vpe/vpdma.c | 36 +- drivers/media/platform/ti-vpe/vpdma.h | 7 + drivers/media/platform/ti-vpe/vpe.c | 251 ++++-- drivers/media/platform/ti-vpe/vpe_regs.h | 187 ----- 10 files changed, 2335 insertions(+), 273 deletions(-) create mode 100644 drivers/media/platform/ti-vpe/csc.c create mode 100644 drivers/media/platform/ti-vpe/csc.h create mode 100644 drivers/media/platform/ti-vpe/sc.c create mode 100644 drivers/media/platform/ti-vpe/sc.h create mode 100644 drivers/media/platform/ti-vpe/sc_coeff.h
For this patch series:
Regards,
Hans
Hi Hans,
Quoted 17 lines · Hans Verkuil
Quoted 6 lines · Archit Taneja
The VPE and VIP IPs in DRA7x contain common scaler and color conversion hardware blocks. We create libraries for these components such that the vpe driver and the vip driver(in future) can use these library funcs to configure the blocks.
There are some points for which I would like comments:
Quoted 9 lines
- For VPE, setting the format and colorspace for the source and destination queues is enough to determine how these blocks need to be configured and whether they need to be bypassed or not. So it didn't make sense to represent them as media controller entities. For VIP(driver not upstream yet), it's possible that there are multiple data paths which may or may not include these blocks. However, the current use cases don't require such flexibility. There may be a need to re-consider a media controller like setup once we work on the VIP driver. Is it a good idea in terms of user-space compatibilty if we use media controller framework in the future.
Quoted 1 line
As long as you don't need the mc, then there is no need to implement it.
The thing is that we want to use these blocks for a future capture hardware called VIP. A VIP port can capture multiple streams from different sensors in time slices, and each of those streams might be sharing the same hardware, but probably in different configurations. I'm not completely aware of media controller details, and whether it can help us here.
I was just wondering if it would be a problem if we later realise that mc might be useful for some use cases. Would implementing it then break previous user space applications which don't call mc ioctls?
Quoted 5 lines
Quoted 5 lines
- These blocks will also require some custom control commands later on. For example, we may want to tell the scaler later on to perform bi-linear scaling, or perform peaking at a particular frequency.
Quoted 7 lines
Quoted 4 lines
- The current series keeps the default scaler coefficients in a header file. These coefficients add a lot of lines of code in the kernel. Does it make more sense for the user application to pass the co-efficients to the kernel using an ioctl? Is there any driver which currenlty does this?
I think it is good to keep it in the driver. Otherwise apps would be forced to set up the table. It's about 11 kilobyte in memory, which isn't that bad.
Okay.
Quoted 34 lines
Quoted 27 lines
The series is based on the branch:
git://linuxtv.org/media_tree.gitmasterArchit Taneja (8):
v4l: ti-vpe: create a scaler block library
v4l: ti-vpe: support loading of scaler coefficients
v4l: ti-vpe: make vpe driver load scaler coefficients
v4l: ti-vpe: enable basic scaler support
v4l: ti-vpe: create a color space converter block library
v4l: ti-vpe: Add helper to perform color conversion
v4l: ti-vpe: enable CSC support for VPE
v4l: ti-vpe: Add a type specifier to describe vpdma data format typedrivers/media/platform/ti-vpe/Makefile | 2 +- drivers/media/platform/ti-vpe/csc.c | 196 +++++ drivers/media/platform/ti-vpe/csc.h | 68 ++ drivers/media/platform/ti-vpe/sc.c | 311 +++++++ drivers/media/platform/ti-vpe/sc.h | 208 +++++ drivers/media/platform/ti-vpe/sc_coeff.h | 1342 ++++++++++++++++++++++++++++++ drivers/media/platform/ti-vpe/vpdma.c | 36 +- drivers/media/platform/ti-vpe/vpdma.h | 7 + drivers/media/platform/ti-vpe/vpe.c | 251 ++++-- drivers/media/platform/ti-vpe/vpe_regs.h | 187 ----- 10 files changed, 2335 insertions(+), 273 deletions(-)Quoted 6 lines
create mode 100644
drivers/media/platform/ti-vpe/csc.ccreate mode 100644drivers/media/platform/ti-vpe/csc.hcreate mode 100644drivers/media/platform/ti-vpe/sc.ccreate mode 100644drivers/media/platform/ti-vpe/sc.hcreate mode 100644drivers/media/platform/ti-vpe/sc_coeff.h
Quoted 3 lines
For this patch series:
Acked-by Hans Verkuil <[email protected]>
Thanks for the review.
Archit
Quoted 33 lines · Archit Taneja
Hi Hans,
Quoted 20 lines · Hans Verkuil
Quoted 7 lines · Archit Taneja
The VPE and VIP IPs in DRA7x contain common scaler and color conversion hardware blocks. We create libraries for these components such that the vpe driver and the vip driver(in future) can use these library funcs to configure the blocks.
There are some points for which I would like comments:
Quoted 11 lines
- For VPE, setting the format and colorspace for the source and destination queues is enough to determine how these blocks need to be configured and whether they need to be bypassed or not. So it didn't make sense to represent them as media controller entities. For VIP(driver not upstream yet), it's possible that there are multiple data paths which may or may not include these blocks. However, the current use cases don't require such flexibility. There may be a need to re-consider a media controller like setup once we work on the VIP driver. Is it a good idea in terms of user-space compatibilty if we use media controller framework in the future.
Quoted 2 lines
As long as you don't need the mc, then there is no need to implement it.
The thing is that we want to use these blocks for a future capture hardware called VIP. A VIP port can capture multiple streams from different sensors in time slices, and each of those streams might be sharing the same hardware, but probably in different configurations. I'm not completely aware of media controller details, and whether it can help us here.
Quoted 4 lines
I was just wondering if it would be a problem if we later realise that mc might be useful for some use cases. Would implementing it then break previous user space applications which don't call mc ioctls?
My understanding is that in the current vpe driver the mc won't be needed, so there is no point adding it. When implementing the vip capture driver the mc might be needed, but that should not require the vpe to add the mc API as well. It's a decision that has to be made per driver.
When you start work on the vip driver it is probably a good idea to talk to Laurent and myself first to see whether the mc is needed or not.
If you have a block diagram of the video hardware that you can share, then that will be quite useful.
Regards,
Hans
Hi,
Quoted 39 lines · Hans Verkuil
Quoted 33 lines · Archit Taneja
Hi Hans,
Quoted 20 lines · Hans Verkuil
Quoted 7 lines · Archit Taneja
The VPE and VIP IPs in DRA7x contain common scaler and color conversion hardware blocks. We create libraries for these components such that the vpe driver and the vip driver(in future) can use these library funcs to configure the blocks.
There are some points for which I would like comments:
Quoted 11 lines
- For VPE, setting the format and colorspace for the source and destination queues is enough to determine how these blocks need to be configured and whether they need to be bypassed or not. So it didn't make sense to represent them as media controller entities. For VIP(driver not upstream yet), it's possible that there are multiple data paths which may or may not include these blocks. However, the current use cases don't require such flexibility. There may be a need to re-consider a media controller like setup once we work on the VIP driver. Is it a good idea in terms of user-space compatibilty if we use media controller framework in the future.
Quoted 2 lines
As long as you don't need the mc, then there is no need to implement it.
The thing is that we want to use these blocks for a future capture hardware called VIP. A VIP port can capture multiple streams from different sensors in time slices, and each of those streams might be sharing the same hardware, but probably in different configurations. I'm not completely aware of media controller details, and whether it can help us here.
Quoted 4 lines
I was just wondering if it would be a problem if we later realise that mc might be useful for some use cases. Would implementing it then break previous user space applications which don't call mc ioctls?
Quoted 5 lines
My understanding is that in the current vpe driver the mc won't be needed, so there is no point adding it. When implementing the vip capture driver the mc might be needed, but that should not require the vpe to add the mc API as well. It's a decision that has to be made per driver.
That's right, vpe doesn't need mc. It might be needed for vip. The reason I brought it up now is because some of the blocks like SC/CSC are replicated in VIP hardware, and I created them in a 'library' sort of way in this series. If vip driver uses mc, these blocks might need to become media entities.
Quoted 4 lines
When you start work on the vip driver it is probably a good idea to talk to Laurent and myself first to see whether the mc is needed or not.
Thanks, that'll be quite useful. I'll look up some mc documentation and drivers using mc myself as well.
Quoted 3 lines
If you have a block diagram of the video hardware that you can share, then that will be quite useful.
Thanks for the clarification. I don't think the DRA7x documentation is public yet. I'll try to look for a block diagram(or create one) and share it with the list.
Archit