Skip to contentSkip to conversation

Seeking guidance for Adding ISP Driver for RK3568

DISCUSSION Yogesh Hegde 2 participants 3 messages Jul 11, 2023 – Jul 12, 2023 Arm Rockchip

lore
Share & tools
lore
#0 lore raw

Hi,

I am seeking your expert guidance and assistance in adding the ISP driver for RK3568 to the Linux Kernel.

Before I start, I have a few questions regarding the process of adding the ISP driver for RK3568 and upstreaming it to the Linux Kernel. I greatly appreciate any insights you can provide on the following points:

1. Integration with existing RK3399 ISP driver: Since the ISP driver for RK3399 has already been upstreamed, I would like to understand if the ISP driver for RK3568 will be added to the existing RK3399 ISP driver or if it will be a completely new driver ?

2. Upstreaming process and staging: I would like to know if the upstreaming process for the RK3568 ISP driver will involve starting with the downstream driver and adding it to the staging area. From there, would it be merged into the media subsystem or follow a different process altogether?

3. Permission requirements from Rockchip: To ensure a smooth upstreaming process, I would like to know if I need any permissions or approvals from Rockchip, the manufacturer of the RK3568 chipset. If permissions are required, could you please guide me on how to obtain them?

I would also like to mention that while this is my first time upstreaming a full driver, I do have some experience working with the media subsystem. I am willing to devote the necessary time and effort to learn and contribute effectively to the upstreaming process also I have Rock 3A development board, which will be helpful in testing the ISP driver.

Thank you very much for considering my request. I eagerly await your responses and any direction you and the mailing list members can provide.

Thanks & Regards,
Yogesh

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

#1 lore raw

Hi Yogesh,

Cc Laurent as I had a chat with him about 1.
Quoted 5 lines · Yogesh Hegde

Hi,

I am seeking your expert guidance and assistance in adding the ISP driver for RK3568 to the Linux Kernel.

Quoted 8 lines

Before I start, I have a few questions regarding the process of adding the ISP driver for RK3568 and upstreaming it to the Linux Kernel. I greatly appreciate any insights you can provide on the following points:

1. Integration with existing RK3399 ISP driver: Since the ISP driver for RK3399 has already been upstreamed, I would like to understand if the ISP driver for RK3568 will be added to the existing RK3399 ISP driver or if it will be a completely new driver ?

That's a very good question, but unfortunately there is no clear answer to it. We were able to hack the existing mainline rkisp1 driver so that it works with the RK3568 ISP (rkisp v2.1), but the result is far from being mainline material.

I would say that there is enough overlap that the RK3568 ISP code should enter drivers/media/platform/rockchip/rkisp1/ but on the other hand the differences are so significant that there should be separate rkisp2-{csi,params,...}.c files.

Example for overlap: With some refactoring applied, rkisp1-capture.c could support the RK3568 ISP as well.

Example for significant changes: the CSI subdevice is fundamentally different, so most likely there will be a rkisp2-csi.c.

Example for a mixture of both: a new stats/params interface to user space will be required for the RK3568 ISP, but if this is designed to be general enough, it might be reused for the older ISP versions at some point.

Quoted 4 lines

2. Upstreaming process and staging: I would like to know if the upstreaming process for the RK3568 ISP driver will involve starting with the downstream driver and adding it to the staging area. From there, would it be merged into the media subsystem or follow a different process altogether?

I think the tendency is towards skipping staging, but I don't have a say on that one.

Quoted 4 lines

3. Permission requirements from Rockchip: To ensure a smooth upstreaming process, I would like to know if I need any permissions or approvals from Rockchip, the manufacturer of the RK3568 chipset. If permissions are required, could you please guide me on how to obtain them?

Downstream code is GPL'ed, so it should be fine.

Quoted 2 lines

I would also like to mention that while this is my first time upstreaming a full driver, I do have some experience working with the media subsystem.

Cool! Just be aware that you picked a complex beast for your first full driver.

Quoted 3 lines

I am willing to devote the necessary time and effort to learn and contribute effectively to the upstreaming process also I have Rock 3A development board, which will be helpful in testing the ISP driver.

What camera sensor hardware do you have?

Best regards,
Michael

Quoted 11 lines

Thank you very much for considering my request. I eagerly await your responses and any direction you and the mailing list members can provide.

Thanks & Regards,
Yogesh

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

#2 lore raw

Hi Michael,

Quoted 52 lines · Michael Riesch

Hi Yogesh,

Cc Laurent as I had a chat with him about 1.
Quoted 5 lines · Yogesh Hegde

Hi,

I am seeking your expert guidance and assistance in adding the ISP driver for RK3568 to the Linux Kernel.

Quoted 8 lines

Before I start, I have a few questions regarding the process of adding the ISP driver for RK3568 and upstreaming it to the Linux Kernel. I greatly appreciate any insights you can provide on the following points:

1. Integration with existing RK3399 ISP driver: Since the ISP driver for RK3399 has already been upstreamed, I would like to understand if the ISP driver for RK3568 will be added to the existing RK3399 ISP driver or if it will be a completely new driver ?

That's a very good question, but unfortunately there is no clear answer to it. We were able to hack the existing mainline rkisp1 driver so that it works with the RK3568 ISP (rkisp v2.1), but the result is far from being mainline material.

I would say that there is enough overlap that the RK3568 ISP code should enter drivers/media/platform/rockchip/rkisp1/ but on the other hand the differences are so significant that there should be separate rkisp2-{csi,params,...}.c files.

Example for overlap: With some refactoring applied, rkisp1-capture.c could support the RK3568 ISP as well.

Example for significant changes: the CSI subdevice is fundamentally different, so most likely there will be a rkisp2-csi.c.

Example for a mixture of both: a new stats/params interface to user space will be required for the RK3568 ISP, but if this is designed to be general enough, it might be reused for the older ISP versions at some point.

Quoted 4 lines

2. Upstreaming process and staging: I would like to know if the upstreaming process for the RK3568 ISP driver will involve starting with the downstream driver and adding it to the staging area. From there, would it be merged into the media subsystem or follow a different process altogether?

I think the tendency is towards skipping staging, but I don't have a say on that one.

Quoted 4 lines

3. Permission requirements from Rockchip: To ensure a smooth upstreaming process, I would like to know if I need any permissions or approvals from Rockchip, the manufacturer of the RK3568 chipset. If permissions are required, could you please guide me on how to obtain them?

Quoted 8 lines

Downstream code is GPL'ed, so it should be fine.

Quoted 2 lines

I would also like to mention that while this is my first time upstreaming a full driver, I do have some experience working with the media subsystem.

Cool! Just be aware that you picked a complex beast for your first full driver.

Thank you for taking the time to respond and provide clarity regarding the process.

Quoted 1 line

What camera sensor hardware do you have?

I have Raspi Cam v1.3 (OV5647) and Raspberry Pi Camera module 2 (IMX219). From the Rock 3A schematics, the MIPI CSI port is Raspberry Pi compatible.

Thanks and Regards
Yogesh

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip