Skip to contentSkip to conversation

Memory sharing issue by application on V4L2 based device driver with system mmu.

DISCUSSION daeinki 5 participants 10 messages Jan 6, 2011 – Jan 13, 2011

Subsystems (2)
lore
Share & tools
lore

daeinki

author
#0 lore raw

Hello, all.

I'd like to discuss memory sharing issue by application on v4l2 based device driver with system mmu and get some advices about that.

Now I am working on Samsung SoC C210 platform and this platform has some multimedia devices with system mmu such as fimc, and mfc also we have implemented device drivers for them. those drivers are based on V4L2 framework with videobuf2. for system mmu of each device, we used VCM(Virtual Contiguous Memory) framework.

Simply, VCM framework provides physical memory, device virtual memory allocation and memory mapping between them. when device driver is initialized or operated by user application, each driver allocates physical memory and device virtual memory and then mapping using VCM interface.

refer to below link for more detail.
http://www.spinics.net/lists/linux-media/msg26548.html

Physical memory access process is as the following.
DVA PA
device --------------> system mmu ------------------> physical memory

DVA : device virtual address.
PA : physical address.

like this, device virtual address should be set to buffer(source or destination) register of multimedia device.

the problem is that application want to share own memory with any device driver to avoid memory copy. in other words, user-allocated memory could be source or destination memory of multimedia device driver.

let's see the diagram below.

                user application

                      |
                      |
                      |
                      |
                      |  1. UVA(allocated by malloc)
                      |
                      |
                    ?|/                   2. UVA(in page unit)

        -----> multimedia device driver -------------------> videobuf2
        |
        |        |     ^                                         |
        |        |     |                                         |
        |        |     -------------------------------------------
        |        |                    3. PA(in page unit)
        |        |
        |        | 4. PA(in page unit)
6. DVA  |        |
        |        |
        |        |
        |      ?|/
        |
        |       Virtual Contiguous Memory ---------
        |                                         |
        |           |     ^                       |
        |           |     |                       | 5. map PA to DVA
        |           |     |                       |
        |           |     |                       |
        -------------     -------------------------

PA : physical address.
UVA : user virtual address.
DVA : device virtual address.

1. user application allocates user space memory through malloc function and sending it to multimedia device driver based on v4l2 framework through userptr feature.

2, 3. multimedia device driver gets translated physical address from videobuf2 framework in page unit.

4, 5. multimedia device driver gets allocated device virtual address and mapping it to physical address and then mapping them through VCM interface.

6. multimedia device driver sets device virtual address from VCM to buffer register.

the diagram above is fully theoretical so I wonder that this way is reasonable and has some problems also what should be considered.

thank you for your interesting.

#1 lore raw

Hello, all.

I'd like to discuss memory sharing issue by application on v4l2 based device driver with system mmu and get some advices about that.

Now I am working on Samsung SoC C210 platform and this platform has some multimedia devices with system mmu such as fimc, and mfc also we have implemented device drivers for them. those drivers are based on V4L2 framework with videobuf2. for system mmu of each device, we used VCM(Virtual Contiguous Memory) framework.

Simply, VCM framework provides physical memory, device virtual memory allocation and memory mapping between them. when device driver is initialized or operated by user application, each driver allocates physical memory and device virtual memory and then mapping using VCM interface.

refer to below link for more detail.
http://www.spinics.net/lists/linux-media/msg26548.html

Physical memory access process is as the following.
DVA PA
device --------------> system mmu ------------------> physical memory

DVA : device virtual address.
PA : physical address.

like this, device virtual address should be set to buffer(source or destination) register of multimedia device.

the problem is that application want to share own memory with any device driver to avoid memory copy. in other words, user-allocated memory could be source or destination memory of multimedia device driver.

let's see the diagram below.

               user application

                     |
                     |
                     |
                     |
                     |  1. UVA(allocated by malloc)
                     |
                     |
                   \|/                   2. UVA(in page unit)

       -----> multimedia device driver -------------------> videobuf2
       |
       |        |     ^                                         |
       |        |     |                                         |
       |        |     -------------------------------------------
       |        |                    3. PA(in page unit)
       |        |
       |        | 4. PA(in page unit)
6. DVA  |        |
       |        |
       |        |
       |      \|/
       |
       |       Virtual Contiguous Memory ---------
       |                                         |
       |           |     ^                       |
       |           |     |                       | 5. map PA to DVA
       |           |     |                       |
       |           |     |                       |
       -------------     -------------------------

PA : physical address.
UVA : user virtual address.
DVA : device virtual address.

1. user application allocates user space memory through malloc function and sending it to multimedia device driver based on v4l2 framework through userptr feature.

2, 3. multimedia device driver gets translated physical address from videobuf2 framework in page unit.

4, 5. multimedia device driver gets allocated device virtual address and mapping it to physical address and then mapping them through VCM interface.

6. multimedia device driver sets device virtual address from VCM to buffer register.

the diagram above is fully theoretical so I wonder that this way is reasonable and has some problems also what should be considered.

thank you for your interesting.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#2 lore raw

Hello,

There are two reasons why malloc isn't suitable for it.

The first is that malloc doesn't allocate memory when malloc is called. So driver or vb2 cannot find PFN for it in the VIDIOC_QBUF.

The second is that malloc uses 4KB page allocation. SYS.MMU(IO-MMU) can handle scattered memory. But it has a penalty when TLB miss is occurred.
So as possible as physically contiguous pages are needed for performance enhancement.

So new allocator which can clear two main issues is needed.

Best regards,

Quoted 6 lines

-----Original Message-----
From: [email protected] [mailto:linux-media- [email protected]] On Behalf Of InKi Dae
Sent: Thursday, January 06, 2011 10:25 PM
To: [email protected]
Subject: Memory sharing issue by application on V4L2 based device driver

with

Quoted 5 lines

system mmu.

Hello, all.

I'd like to discuss memory sharing issue by application on v4l2 based

device driver

Quoted 5 lines

with system mmu and get some advices about that.

Now I am working on Samsung SoC C210 platform and this platform has some multimedia devices with system mmu such as fimc, and mfc also we have implemented device drivers for them. those drivers are based on V4L2

framework

Quoted 1 line

with videobuf2. for system mmu of each device, we used VCM(Virtual

Contiguous

Quoted 4 lines

Memory) framework.

Simply, VCM framework provides physical memory, device virtual memory allocation and memory mapping between them. when device driver is

initialized or

Quoted 1 line

operated by user application, each driver allocates physical memory and

device

Quoted 9 lines

virtual memory and then mapping using VCM interface.

refer to below link for more detail.
http://www.spinics.net/lists/linux-media/msg26548.html

Physical memory access process is as the following.
DVA PA
device --------------> system mmu ------------------> physical memory

Quoted 7 lines

DVA : device virtual address.
PA : physical address.

like this, device virtual address should be set to buffer(source or destination) register of multimedia device.

the problem is that application want to share own memory with any device

driver to

Quoted 1 line

avoid memory copy. in other words, user-allocated memory could be source

or

Quoted 37 lines

destination memory of multimedia device driver.

let's see the diagram below.

               user application

                     |
                     |
                     |
                     |
                     |  1. UVA(allocated by malloc)
                     |
                     |
                   \|/                   2. UVA(in page unit)

       -----> multimedia device driver -------------------> videobuf2
       |
       |        |     ^                                         |
       |        |     |                                         |
       |        |     -------------------------------------------
       |        |                    3. PA(in page unit)
       |        |
       |        | 4. PA(in page unit)
6. DVA  |        |
       |        |
       |        |
       |      \|/
       |
       |       Virtual Contiguous Memory ---------
       |                                         |
       |           |     ^                       |
       |           |     |                       | 5. map PA to DVA
       |           |     |                       |
       |           |     |                       |
       -------------     -------------------------
Quoted 5 lines

PA : physical address.
UVA : user virtual address.
DVA : device virtual address.

1. user application allocates user space memory through malloc function

and

Quoted 1 line

sending it to multimedia device driver based on v4l2 framework through

userptr

Quoted 6 lines

feature.

2, 3. multimedia device driver gets translated physical address from videobuf2 framework in page unit.

4, 5. multimedia device driver gets allocated device virtual address and

mapping it

Quoted 3 lines

to physical address and then mapping them through VCM interface.

6. multimedia device driver sets device virtual address from VCM to

buffer register.

Quoted 2 lines

the diagram above is fully theoretical so I wonder that this way is

reasonable and

Quoted 10 lines

has some problems also what should be considered.

thank you for your interesting.

_______________________________________________
linux-arm-kernel mailing list [email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in

the body

Quoted 2 lines

of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html

#3 lore raw

thank you for your comments.

your second comment has no any problem as I said before, user virtual addess could be translated in page unit. but the problem, as you said, is that when cpu access to the memory in user mode, the memory allocated by malloc, page fault occurs so we can't find pfn to user virtual address. I missed that. but I think we could resolve this one.

as before, user application allocates memory through malloc function and then send it to device driver(using userptr feature). if the pfn is null when device driver translated user virtual address in page unit then it allocates phsical memory in page unit using some interface such as alloc_page() and then mapping them. when pfn is null, to check it and allocate physical memory in page unit could be processed by videobuf2.

of course, videobuf2 has no any duty considered for system mmu. so videobuf2 just provides callback for 3rd party and any platform with system mmu such as Samsung SoC C210 implements the function(allocating physical memory and mapping it) and registers it to callback of videobuf2. by doing so, I think your first comment could be cleared.

please, feel free to give me your opinion and pointing out.

thank you.

2011? 1? 7? ?? 8:57, Jonghun Han <[email protected]>?? ?:

Quoted 139 lines

Hello,

There are two reasons why malloc isn't suitable for it.

The first is that malloc doesn't allocate memory when malloc is called. So driver or vb2 cannot find PFN for it in the VIDIOC_QBUF.

The second is that malloc uses 4KB page allocation. SYS.MMU(IO-MMU) can handle scattered memory. But it has a penalty when TLB miss is occurred.
So as possible as physically contiguous pages are needed for performance enhancement.

So new allocator which can clear two main issues is needed.

Best regards,

Quoted 6 lines

-----Original Message-----
From: linux-media-owner at vger.kernel.org [mailto:linux-media- owner at vger.kernel.org] On Behalf Of InKi Dae
Sent: Thursday, January 06, 2011 10:25 PM
To: linux-media at vger.kernel.org
Subject: Memory sharing issue by application on V4L2 based device driver

with

Quoted 5 lines

system mmu.

Hello, all.

I'd like to discuss memory sharing issue by application on v4l2 based

device driver

Quoted 5 lines

with system mmu and get some advices about that.

Now I am working on Samsung SoC C210 platform and this platform has some multimedia devices with system mmu such as fimc, and mfc also we have implemented device drivers for them. those drivers are based on V4L2

framework

Quoted 1 line

with videobuf2. for system mmu of each device, we used VCM(Virtual

Contiguous

Quoted 4 lines

Memory) framework.

Simply, VCM framework provides physical memory, device virtual memory allocation and memory mapping between them. when device driver is

initialized or

Quoted 1 line

operated by user application, each driver allocates physical memory and

device

Quoted 9 lines

virtual memory and then mapping using VCM interface.

refer to below link for more detail.
http://www.spinics.net/lists/linux-media/msg26548.html

Physical memory access process is as the following.
DVA PA
device --------------> system mmu ------------------> physical memory

Quoted 7 lines

DVA : device virtual address.
PA : physical address.

like this, device virtual address should be set to buffer(source or destination) register of multimedia device.

the problem is that application want to share own memory with any device

driver to

Quoted 1 line

avoid memory copy. in other words, user-allocated memory could be source

or

Quoted 37 lines

destination memory of multimedia device driver.

let's see the diagram below.

               user application

                     |
                     |
                     |
                     |
                     |  1. UVA(allocated by malloc)
                     |
                     |
                   ?|/                   2. UVA(in page unit)

       -----> multimedia device driver -------------------> videobuf2
       |
       |        |     ^                                         |
       |        |     |                                         |
       |        |     -------------------------------------------
       |        |                    3. PA(in page unit)
       |        |
       |        | 4. PA(in page unit)
6. DVA  |        |
       |        |
       |        |
       |      ?|/
       |
       |       Virtual Contiguous Memory ---------
       |                                         |
       |           |     ^                       |
       |           |     |                       | 5. map PA to DVA
       |           |     |                       |
       |           |     |                       |
       -------------     -------------------------
Quoted 5 lines

PA : physical address.
UVA : user virtual address.
DVA : device virtual address.

1. user application allocates user space memory through malloc function

and

Quoted 1 line

sending it to multimedia device driver based on v4l2 framework through

userptr

Quoted 6 lines

feature.

2, 3. multimedia device driver gets translated physical address from videobuf2 framework in page unit.

4, 5. multimedia device driver gets allocated device virtual address and

mapping it

Quoted 3 lines

to physical address and then mapping them through VCM interface.

6. multimedia device driver sets device virtual address from VCM to

buffer register.

Quoted 2 lines

the diagram above is fully theoretical so I wonder that this way is

reasonable and

Quoted 10 lines

has some problems also what should be considered.

thank you for your interesting.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in

the body

Quoted 2 lines

of a message to majordomo at vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html

#4 lore raw

Hello,

That's not a translation issue. What I mention is the size of allocation. The malloc uses 4KB page allocation and SYS.MMU can handle it. But 64KB or 1MB physically contiguous memory is better than 4KB page in the point of performance.

Best regards,

Quoted 6 lines

-----Original Message-----
From: linux-media-owner at vger.kernel.org [mailto:linux-media- owner at vger.kernel.org] On Behalf Of InKi Dae
Sent: Friday, January 07, 2011 11:17 AM
To: Jonghun Han
Cc: linux-media at vger.kernel.org; linux-arm-kernel at lists.infradead.org;

linux-fbdev;

Quoted 2 lines

kyungmin.park at samsung.com
Subject: Re: Memory sharing issue by application on V4L2 based device

driver

Quoted 5 lines

with system mmu.

thank you for your comments.

your second comment has no any problem as I said before, user virtual

addess

Quoted 1 line

could be translated in page unit. but the problem, as you said, is that

when cpu

Quoted 1 line

access to the memory in user mode, the memory allocated by malloc, page

fault

Quoted 1 line

occurs so we can't find pfn to user virtual address. I missed that. but I

think we

Quoted 3 lines

could resolve this one.

as before, user application allocates memory through malloc function and

then

Quoted 1 line

send it to device driver(using userptr feature). if the pfn is null when

device driver

Quoted 1 line

translated user virtual address in page unit then it allocates phsical

memory in

Quoted 1 line

page unit using some interface such as alloc_page() and then mapping

them. when

Quoted 1 line

pfn is null, to check it and allocate physical memory in page unit could

be

Quoted 4 lines

processed by videobuf2.

of course, videobuf2 has no any duty considered for system mmu. so videobuf2 just provides callback for 3rd party and any platform with

system mmu

Quoted 1 line

such as Samsung SoC C210 implements the function(allocating physical

memory

Quoted 1 line

and mapping it) and registers it to callback of videobuf2. by doing so, I

think your

Quoted 151 lines

first comment could be cleared.

please, feel free to give me your opinion and pointing out.

thank you.

2011? 1? 7? ?? 8:57, Jonghun Han <[email protected]>?? ?:

Quoted 144 lines

Hello,

There are two reasons why malloc isn't suitable for it.

The first is that malloc doesn't allocate memory when malloc is called. So driver or vb2 cannot find PFN for it in the VIDIOC_QBUF.

The second is that malloc uses 4KB page allocation. SYS.MMU(IO-MMU) can handle scattered memory. But it has a penalty when TLB miss is occurred.
So as possible as physically contiguous pages are needed for performance enhancement.

So new allocator which can clear two main issues is needed.

Best regards,

Quoted 7 lines

-----Original Message-----
From: linux-media-owner at vger.kernel.org [mailto:linux-media- owner at vger.kernel.org] On Behalf Of InKi Dae
Sent: Thursday, January 06, 2011 10:25 PM
To: linux-media at vger.kernel.org
Subject: Memory sharing issue by application on V4L2 based device driver

with

Quoted 5 lines

system mmu.

Hello, all.

I'd like to discuss memory sharing issue by application on v4l2 based

device driver

Quoted 6 lines

with system mmu and get some advices about that.

Now I am working on Samsung SoC C210 platform and this platform has some multimedia devices with system mmu such as fimc, and mfc also we have implemented device drivers for them. those drivers are based on V4L2

framework

Quoted 1 line

with videobuf2. for system mmu of each device, we used VCM(Virtual

Contiguous

Quoted 5 lines

Memory) framework.

Simply, VCM framework provides physical memory, device virtual memory allocation and memory mapping between them. when device driver is

initialized or

Quoted 2 lines

operated by user application, each driver allocates physical memory and

device

Quoted 9 lines

virtual memory and then mapping using VCM interface.

refer to below link for more detail.
http://www.spinics.net/lists/linux-media/msg26548.html

Physical memory access process is as the following.
DVA PA
device --------------> system mmu ------------------> physical memory

Quoted 8 lines

DVA : device virtual address.
PA : physical address.

like this, device virtual address should be set to buffer(source or destination) register of multimedia device.

the problem is that application want to share own memory with any device

driver to

Quoted 2 lines

avoid memory copy. in other words, user-allocated memory could be source

or

Quoted 37 lines

destination memory of multimedia device driver.

let's see the diagram below.

               user application

                     |
                     |
                     |
                     |
                     |  1. UVA(allocated by malloc)
                     |
                     |
                   ?|/                   2. UVA(in page unit)

       -----> multimedia device driver -------------------> videobuf2
       |
       |        |     ^                                         |
       |        |     |                                         |
       |        |     -------------------------------------------
       |        |                    3. PA(in page unit)
       |        |
       |        | 4. PA(in page unit)
6. DVA  |        |
       |        |
       |        |
       |      ?|/
       |
       |       Virtual Contiguous Memory ---------
       |                                         |
       |           |     ^                       |
       |           |     |                       | 5. map PA to DVA
       |           |     |                       |
       |           |     |                       |
       -------------     -------------------------
Quoted 6 lines

PA : physical address.
UVA : user virtual address.
DVA : device virtual address.

1. user application allocates user space memory through malloc function

and

Quoted 2 lines

sending it to multimedia device driver based on v4l2 framework through

userptr

Quoted 7 lines

feature.

2, 3. multimedia device driver gets translated physical address from videobuf2 framework in page unit.

4, 5. multimedia device driver gets allocated device virtual address and

mapping it

Quoted 3 lines

to physical address and then mapping them through VCM interface.

6. multimedia device driver sets device virtual address from VCM to

buffer register.

Quoted 2 lines

the diagram above is fully theoretical so I wonder that this way is

reasonable and

Quoted 11 lines

has some problems also what should be considered.

thank you for your interesting.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in

Quoted 7 lines
Quoted 5 lines

the body

Quoted 2 lines

of a message to majordomo at vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in

the body

Quoted 2 lines

of a message to majordomo at vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html

author
#5 lore raw

Hello, Mr. Jonghun.

please, note that user process's pgd and device's pgd with system mmu are differnent. first, the memory allocated by user malloc should be mapped to physical memory in page unit because user app also should be accessed to own memory region. and when mapping the physical memory to device virtual address, it creates page table entry for 64k or 1M and then could map physical address to those entries. therefore what you mention has no problem.

actual issue is not to need demend paging, some issues we don't understand may exist as side effects.

thank you.

Jonghun Han ? ?:

Quoted 216 lines

Hello,

That's not a translation issue. What I mention is the size of allocation. The malloc uses 4KB page allocation and SYS.MMU can handle it. But 64KB or 1MB physically contiguous memory is better than 4KB page in the point of performance.

Best regards,

Quoted 6 lines

-----Original Message-----
From: linux-media-owner at vger.kernel.org [mailto:linux-media- owner at vger.kernel.org] On Behalf Of InKi Dae
Sent: Friday, January 07, 2011 11:17 AM
To: Jonghun Han
Cc: linux-media at vger.kernel.org; linux-arm-kernel at lists.infradead.org;

linux-fbdev;

Quoted 2 lines

kyungmin.park at samsung.com
Subject: Re: Memory sharing issue by application on V4L2 based device

driver

Quoted 5 lines

with system mmu.

thank you for your comments.

your second comment has no any problem as I said before, user virtual

addess

Quoted 1 line

could be translated in page unit. but the problem, as you said, is that

when cpu

Quoted 1 line

access to the memory in user mode, the memory allocated by malloc, page

fault

Quoted 1 line

occurs so we can't find pfn to user virtual address. I missed that. but I

think we

Quoted 3 lines

could resolve this one.

as before, user application allocates memory through malloc function and

then

Quoted 1 line

send it to device driver(using userptr feature). if the pfn is null when

device driver

Quoted 1 line

translated user virtual address in page unit then it allocates phsical

memory in

Quoted 1 line

page unit using some interface such as alloc_page() and then mapping

them. when

Quoted 1 line

pfn is null, to check it and allocate physical memory in page unit could

be

Quoted 4 lines

processed by videobuf2.

of course, videobuf2 has no any duty considered for system mmu. so videobuf2 just provides callback for 3rd party and any platform with

system mmu

Quoted 1 line

such as Samsung SoC C210 implements the function(allocating physical

memory

Quoted 1 line

and mapping it) and registers it to callback of videobuf2. by doing so, I

think your

Quoted 149 lines

first comment could be cleared.

please, feel free to give me your opinion and pointing out.

thank you.

2011? 1? 7? ?? 8:57, Jonghun Han <[email protected]>?? ?:

Quoted 142 lines

Hello,

There are two reasons why malloc isn't suitable for it.

The first is that malloc doesn't allocate memory when malloc is called. So driver or vb2 cannot find PFN for it in the VIDIOC_QBUF.

The second is that malloc uses 4KB page allocation. SYS.MMU(IO-MMU) can handle scattered memory. But it has a penalty when TLB miss is occurred.
So as possible as physically contiguous pages are needed for performance enhancement.

So new allocator which can clear two main issues is needed.

Best regards,

Quoted 7 lines

-----Original Message-----
From: linux-media-owner at vger.kernel.org [mailto:linux-media- owner at vger.kernel.org] On Behalf Of InKi Dae
Sent: Thursday, January 06, 2011 10:25 PM
To: linux-media at vger.kernel.org
Subject: Memory sharing issue by application on V4L2 based device driver

with

Quoted 5 lines

system mmu.

Hello, all.

I'd like to discuss memory sharing issue by application on v4l2 based

device driver

Quoted 6 lines

with system mmu and get some advices about that.

Now I am working on Samsung SoC C210 platform and this platform has some multimedia devices with system mmu such as fimc, and mfc also we have implemented device drivers for them. those drivers are based on V4L2

framework

Quoted 1 line

with videobuf2. for system mmu of each device, we used VCM(Virtual

Contiguous

Quoted 5 lines

Memory) framework.

Simply, VCM framework provides physical memory, device virtual memory allocation and memory mapping between them. when device driver is

initialized or

Quoted 2 lines

operated by user application, each driver allocates physical memory and

device

Quoted 9 lines

virtual memory and then mapping using VCM interface.

refer to below link for more detail.
http://www.spinics.net/lists/linux-media/msg26548.html

Physical memory access process is as the following.
DVA PA
device --------------> system mmu ------------------> physical memory

Quoted 8 lines

DVA : device virtual address.
PA : physical address.

like this, device virtual address should be set to buffer(source or destination) register of multimedia device.

the problem is that application want to share own memory with any device

driver to

Quoted 2 lines

avoid memory copy. in other words, user-allocated memory could be source

or

Quoted 37 lines

destination memory of multimedia device driver.

let's see the diagram below.

               user application

                     |
                     |
                     |
                     |
                     |  1. UVA(allocated by malloc)
                     |
                     |
                   ?|/                   2. UVA(in page unit)

       -----> multimedia device driver -------------------> videobuf2
       |
       |        |     ^                                         |
       |        |     |                                         |
       |        |     -------------------------------------------
       |        |                    3. PA(in page unit)
       |        |
       |        | 4. PA(in page unit)
6. DVA  |        |
       |        |
       |        |
       |      ?|/
       |
       |       Virtual Contiguous Memory ---------
       |                                         |
       |           |     ^                       |
       |           |     |                       | 5. map PA to DVA
       |           |     |                       |
       |           |     |                       |
       -------------     -------------------------
Quoted 6 lines

PA : physical address.
UVA : user virtual address.
DVA : device virtual address.

1. user application allocates user space memory through malloc function

and

Quoted 2 lines

sending it to multimedia device driver based on v4l2 framework through

userptr

Quoted 7 lines

feature.

2, 3. multimedia device driver gets translated physical address from videobuf2 framework in page unit.

4, 5. multimedia device driver gets allocated device virtual address and

mapping it

Quoted 3 lines

to physical address and then mapping them through VCM interface.

6. multimedia device driver sets device virtual address from VCM to

buffer register.

Quoted 1 line

the diagram above is fully theoretical so I wonder that this way is

reasonable and

Quoted 11 lines

has some problems also what should be considered.

thank you for your interesting.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in

Quoted 6 lines
Quoted 4 lines

the body

Quoted 2 lines

of a message to majordomo at vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in

the body

Quoted 2 lines

of a message to majordomo at vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw
In reply to InKi Dae
Quoted 16 lines · InKi Dae

thank you for your comments.

your second comment has no any problem as I said before, user virtual addess could be translated in page unit. but the problem, as you said, is that when cpu access to the memory in user mode, the memory allocated by malloc, page fault occurs so we can't find pfn to user virtual address. I missed that. but I think we could resolve this one.

as before, user application allocates memory through malloc function and then send it to device driver(using userptr feature). if the pfn is null when device driver translated user virtual address in page unit then it allocates phsical memory in page unit using some interface such as alloc_page() and then mapping them. when pfn is null, to check it and allocate physical memory in page unit could be processed by videobuf2.

Quoted 5 lines

of course, videobuf2 has no any duty considered for system mmu. so videobuf2 just provides callback for 3rd party and any platform with system mmu such as Samsung SoC C210 implements the function(allocating physical memory and mapping it) and registers it to callback of videobuf2. by doing so, I think your first comment could be cleared.

FWIW, TTM (drivers/gpu/drm/ttm, include/drm/ttm) is designed and used for managing memory between CPU/GPU and kernel/userspace access. I haven't looked at your requirements in detail, but if you haven't looked at TTM yet, it sounds like it might be worth a look.

Signature · 2 lines
Earthling Michel D?nzer           |                http://www.vmware.com
Libre software enthusiast         |          Debian, X and DRI developer
author
#9 lore raw

Michel D?nzer ? ?:

Quoted 23 lines · InKi Dae
Quoted 16 lines

thank you for your comments.

your second comment has no any problem as I said before, user virtual addess could be translated in page unit. but the problem, as you said, is that when cpu access to the memory in user mode, the memory allocated by malloc, page fault occurs so we can't find pfn to user virtual address. I missed that. but I think we could resolve this one.

as before, user application allocates memory through malloc function and then send it to device driver(using userptr feature). if the pfn is null when device driver translated user virtual address in page unit then it allocates phsical memory in page unit using some interface such as alloc_page() and then mapping them. when pfn is null, to check it and allocate physical memory in page unit could be processed by videobuf2.

Quoted 5 lines

of course, videobuf2 has no any duty considered for system mmu. so videobuf2 just provides callback for 3rd party and any platform with system mmu such as Samsung SoC C210 implements the function(allocating physical memory and mapping it) and registers it to callback of videobuf2. by doing so, I think your first comment could be cleared.

Quoted 6 lines

FWIW, TTM (drivers/gpu/drm/ttm, include/drm/ttm) is designed and used for managing memory between CPU/GPU and kernel/userspace access. I haven't looked at your requirements in detail, but if you haven't looked at TTM yet, it sounds like it might be worth a look.

Thank you for your advice. I would look at TTM driver.

#6 lore raw

Re: Memory sharing issue by application on V4L2 based device

In reply to Jonghun Han

.. snip..

Quoted 2 lines

But 64KB or 1MB physically contiguous memory is better than 4KB page in the point of performance.

Could you explain that in more details please? I presume you are talking about a CPU that has a MMU unit, right?

#7 lore raw

Sorry for late reply.

Samsung SoC C210 has many multimedia IPs.
Each IP has its own MMU named SYSTEM MMU like CPU's MMU.
So it can handle discontiguous memory using device own virtual address.

What Inki Dae wants to discuss is how to allocate the memory and how to share it with other multimedia IPs.

Thank you for interesting.

Best regards,

Quoted 6 lines

-----Original Message-----
From: linux-fbdev-owner at vger.kernel.org [mailto:linux-fbdev- owner at vger.kernel.org] On Behalf Of Konrad Rzeszutek Wilk
Sent: Wednesday, January 12, 2011 2:35 AM
To: Jonghun Han
Cc: 'InKi Dae'; linux-media at vger.kernel.org;

linux-arm-kernel at lists.infradead.org;

Quoted 2 lines

'linux-fbdev'; kyungmin.park at samsung.com
Subject: Re: Memory sharing issue by application on V4L2 based device

driver

Quoted 8 lines

with system mmu.

.. snip..

Quoted 2 lines

But 64KB or 1MB physically contiguous memory is better than 4KB page in the point of performance.

Could you explain that in more details please? I presume you are talking

about a

Quoted 3 lines

CPU that has a MMU unit, right?
--
To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in

the body

Quoted 2 lines

of a message to majordomo at vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html