Multimedia-chat-meeting-2016-04-19 goooooood morning ! [15:54] morning or afternoon hello [15:55] hi guys hi we have everybody, perfect [15:57] let's get started (sorry, I was opening the tasks list) agenda for today Topic 1. Status check for the multimedia tasks [15:58] Topic 2. Additional '50%' tasks Topic 3. Next meeting anything else ? I'll take that as a no [15:59] Topic 1. Status check for the multimedia tasks let's handle that in alphabetical order [16:00] we have, for Magnus VIN,?,plan,magnus,IPMMU integration on Gen2 VIN,?,plan,magnus,IPMMU integration on Gen3 VIN,?,plan,magnus,IPMMU support on Gen2 VIN,?,plan,magnus,IPMMU support on Gen3 no progress so far =) ok, that's an easy one :-) [16:01] now, for Morimoto-san RSND,2016-06-30,plan,morimoto,HDMI sound Prototype on Gen3 RSND,2016-06-30,plan,morimoto,HDMI sound Upstream support without hotplug on Gen2 RSND,2016-09-30,plan,morimoto,Hotplug support upstream on Gen3 I'm fighting with them I noticed it can be 3 step 1. DT graph [16:02] 2. HDMI + ASoC 3. RSND support I sent prototype patch for 1. and got any response I've seen that. would you like me to review it ? Thanks, but it seems OK now [16:03] I think I understand graph concept but, not yet understand motivation *** horms_ (~horms@124-171-1-229.dyn.iinet.net.au) has joined channel #periperi that I can explain please [16:04] it came from V4L2 Yeah, I know. my motivation means complex camera devices are made of multiple hardware devices camera sensor, CSI receiver, ISP, lens controller, flash controller, ... those devices can be inside the SoC or on-board [16:05] and they can be mixed and matched on different SoCs/boards so we have, in V4L2, a framework to support them using multiple drivers it's a bit similar to ASoC when we had to create DT bindings we needed a way to express how the devices were connected together from a data flow point of view like saying that the camera sensor output is connected to the input of CSI-2 receiver 2 [16:06] or that the output of CSI-2 receiver 0 is connected to input 3 of the ISP essentially, that's expressing a graph of data flow connections hence the OF graph bindings [16:07] a device represented in DT can have multiple ports those are connection points through which data flows and to represent connections between ports, we use endpoints and endpoint repreent one end of a connection [16:08] we need the endpoints because a port on one DT node can be connected to ports of multiple other DT nodes is that clearer ? 80% [16:09] I understand that the connection is very complex. but my question is pinchartl: do the ports map directly to hardware? yes they do so do the connections morimoto-san was saying? [16:10] my question is 1) we can happy to find-out its connection when probing timing ? 2) does graph supports runtime switching ?? do we would like to find its connection on probe timing ?? we can use phandle for connection itself ? [16:11] (for 1. question) I'm not sure to understand the first question I think phandle can do that ? [16:12] if we would like to know "connection" we use phandles to describe connections a connection is a link between two ports it's represented as a phandle to an endpoint [16:13] the endpoint on one side of the connection references the endpoint on the other side through a phandle, with the remote-endpoint property Is this means, driver want to know who is connected, on probe time ? [16:14] or which channel is used ? that's what drivers usually do note that the connections in DT describe the possible hardware connections [16:15] not the active connections at a given point of time DT will tell that device A and device B are connected at the hardware level it will not tell whether the connection is currently in use so DT will show all possible connections "all possible connections" [16:16] is this the purpose of graph ? the purpose of the graph is to describe how the hardware pieces are connected together that's all Hmm... OK, and it indicate connection only [16:17] can't do runtime switch it certainly doesn't handle that, no you can't modify connections at runtime [16:18] OK pinchartl: if you want to perform a run time switch, what do you use? unless you solder new wires to the board to create new connections but I think that's out of scope :-) runtime activation or deactivation of a connection is left for the driver to handle, it doesn't involve DT DT describes the hardware * geertu is dreaming of DT overlays... [16:19] sure, i understand it describes how the data signals are routed on the board or in the SoC but how does the typical driver perform activation/deactivtation? it's subsystem-specific [16:21] ok i see thanks even the concept of connection activation/deactivtation? is subsystem-specific my patch was super quick hack for ASoC, not good for full-graph. my patch reviewer said that current ASoC style is not good match to graph style it's not defined by the DT bindings who reviewed it ? Jean-Francois Moine [16:22] did it seems he is working for this graph + ASoC from last year did Lars comment too ? No comment from Lars graph + ASoC patch was posted last year, but not accepted [16:23] and he said (if my understanding was correct) it need different style [16:24] total in total I couldn't understand what is the next step for graph + ASoC [16:25] did he explain what different style ? Current ASoC is using CPU driver + Codec driver + Card driver but (if my understanding was correct) [16:26] graph + ASoC doens't need Card driver And CPU driver needs update in my understand that's correct I believe [16:27] I need more investigate this topic DT point <-> Code point I think this is the reason why ASoC had Card driver... [16:28] OK, this is current my DT issue yes, the card driver is similar in purpose to the OF graph DT bindings [16:29] Ahh, OK, this is understandable comment for me :) I couldn't 100% understand why graph is needed, because ASoC already had Card :) [16:30] And, next 2. HDMI + ASoC the idea is that video uses the OF graph DT bindings and alsa has the ASoC DT bindings and now that we have a connector that can carry both video and audio, we need to connect the two that's the issue two totally different DT binding concepts that need to be connected together [16:31] I think it is not easy above Jean-Francois had tring this issue but not yet accepted [16:32] and next HDMI sound + ASoC, I know Russel added dw-hdmi-ahb-audio driver [16:34] I think DU is using dw-hdmi-bind() function above is related to it. I think this driver seems have compatible [16:35] but, the data flows is not same as our chip and our datasheet doesn't indicate detail dw-hdmi-ahb-audio is using mem -> DMA -> chip [16:36] there's no proper upstream solution at this point. Russell sometimes has, let's say, very personal and peculiar ideas when it comes to DT bindings or driver framework designs our case, mem -> SSI -> chip by DMA do you mean DMA engine ? not DMAEngine, it seems this chip has DMA inside [16:37] as in the Linux DMA engine API ok dw-hdmi-ahb-audio might need to be reworked but our datasheet doesn't show it I haven't looked at it, I don't know how it works it seems this chip is supporting many data transfer style anyway, I'm asking detail to HW team [16:38] ok regarding the OF graph DT bindings we need to solve the problem of two separate DT bindings existing for video and audio I don't know what will be accepted upstream [16:39] it's an open issue, nobody has solved it upstream yet yeah I think it is not easy ? so I'm afraid this needs new designs, new ideas, and lots of discussion to agree on a solution with everybody it's certainly not easy, no [16:40] Do you think this is 1st priority ? Or should I create prototype as 1st step ? DT bindings are a stable ABI, so they are a prerequisite for upstream merge prototype = HDMI can use sound somehow you can certainly create a prototype [16:41] if we want to convince people about a given DT bindings scheme we have to show that it works, and how it works so an implementation is needed Sorry, my prototype means skip DT at this point, and work to HDMI sound output as 1st [16:42] you can do that, but it can't be merged upstream without DT bindings Yes, prototype is needed to discussion Yes. So, this DT issue is my 1st priority [16:43] DT bindings take time to be discussed and agreed on so it makes sense to start early *** wsa_ (~wsa@p4FE25DE7.dip0.t-ipconnect.de) has joined channel #periperi OK, will try but while discussions are ongoing, a prototype of sound support can certainly be written [16:44] the runtime side doesn't depend on DT only probind does I think DT is big issu [16:45] issue HDMI + ASoC is big issue SSI HDMI supoort is not a big issue I think I agree, although changes to the dw-hdmi-ahb-audio driver might not be very easy to get past Russell [16:46] OK, thanks [16:47] this is current my headache, oops, progress :-) so what's your plan ? create a prototype in parallel to the DT bindings discussions ? [16:48] parallel ? Yah yes at the same time DT bindings discussions + HDMI sound prototype code, do you mean ? [16:49] yes yes Can you separate todo list for me ? [16:51] DT + dw-hdmi-ahb-audio + SSI split "RSND,2016-06-30,plan,morimoto,HDMI sound Prototype on Gen3" in three ? Yes [16:52] ow about RSND,2016-06-30,plan,morimoto,DT bindings for HDMI sound RSND,2016-06-30,plan,morimoto,dw-hdmi-ahb-audio prototype on Gen3 RSND,2016-06-30,plan,morimoto,SSI prototype on Gen3 (s/ow/how/) SSI prototype -> SSI + HDMI support prototype [16:53] is understandable. other 2 are nice for me ok I'll do that thanks anything else for audio ? nothing ok [16:54] next, Niklas ADV7482,v4.7,plan,niklas,Prototype on Gen3 ADV7482,v4.8,plan,niklas,Gen3 support upstream ADV7482,v4.8,plan,niklas,Interlace support upstream VIN,?,plan,niklas,Gen3 support VIN,?,plan,niklas,Scaler support (on Gen3) VIN,v4.7,plan,niklas,CSI2 prototype (Gen3) VIN,v4.7,public,niklas,New VIN driver without soc-camera (tested on Gen2) VIN,v4.8,plan,niklas,CSI2 interlace support upstream (Gen3) VIN,v4.8,plan,niklas,CSI2 support upstream (Gen3) VIN,v4.8,plan,niklas,Gen3 support upstream (without CSI-2) VIN on Gen2 on track, will need another patch Hans extended the test suite so minior work left [16:55] :-) started on VIN on Gen3, prorted the adv7482 and csi2 from BSP and it compiles [16:56] had a fight with cpg but think I understand it now so next is to fix up DT and update rcar-vin to support multiple subdevs [16:57] ok do you know how you will do that, or do you need guidance ? [16:58] current problem is to figure out for which subdevice calls i need to use v4l2_device_call_until_err instead of v4l2_subdev_call but I have not yet read the code so it might be obvious then as you're limited to two subdevices, v4l2_device_call_until_err() might not be the best choice [16:59] at least not for all operations some of them will likely need to be called manually for things like s_stream you will likely want to control the order [17:00] call s_stream(1) on the CSI-2 receiver first, and then on the ADV7482 as you're limited to two subdevs, and one of them is internal to the SoC, custom code might be a better option than trying to be too generic [17:01] ok so you think its best not to use a list for subdevs but instead make explicit struct variables for them since I'm limited to only two? I think so, yes [17:02] we can revisit that later if needed, if we need to support more than two subdevices ok good input, I was looking at the xilinx driver but I can agree to that it's a bit over complex for only two subdevs [17:03] but I can almost guarantee that any generic code you write now with two subdevices as the only test case won't be generic enough :-) :) it's a very complex problem, let's not try to solve it now or even ever, at least until we need to ok then I feel I'm on track. Will focus on Gen3 VIN first then CSI2 and last ADV7482 [17:04] perfect anything else ? nope I think I'm good great thanks next, me [17:05] DU,?,plan,laurent,DU+VSPD Integration in Renesas drivers (Gen3) DU,?,plan,laurent,IPMMU integration on Gen3 DU,?,plan,laurent,IPMMU support on Gen3 (through VSPD+FCP) DU,v4.7,public,laurent,VSPD Z-order support upstream (Gen3) a large patch series for the VSP driver, needed for DU support, was accepted upstream [17:06] I'll rebase the integration patches and resubmit them geertu: if you're here, is the power domain patch on track ? when do you expect it to be merged ? anyway, I'll rebase and resubmit [17:08] pinchartl: I have to integrate the small comments from Ulf, push out a stable branch for the clock people (who didn't comment so far) that Simon can pull, too geertu: thanks when do you expect it to be merged ? pinchartl: After that Simon can apply all of it. I'll do the branch tomorrow pincahrtl: If everything goes well, the end of the week? (depends on clock people) [17:09] \o/ that would be very nice regarding IPMMU integration, no progress Z-order support is public, I'll send a pull request [17:10] pinchartl: BSP team is very waiting Request API Test Program. when can we receive it ? morimoto: let me continue going through the list, I'll address that [17:11] VSP,?,plan,laurent,Fixed alpha support (VI6_DPR_*_ROUTE.FXA) VSP,?,plan,laurent,UDS regression fix VSP,v4.8,plan,laurent,Fix suspend/resume crash VSP,?,plan,laurent,CLU WARN_ON fix VSP,?,plan,laurent,CLU 2D and 3D mode support VSP,?,plan,laurent,CLU/LUT test application VSP,?,plan,laurent,CLU/LUT upstream API ok none of that has been started. suspend/resume and CLU are covered by additional tasks [17:12] there's VSP,v4.7,public,laurent,CLU/LUT support submitted upstream on Gen3 too that's also covered by an additional task for June, so it will be in v4.8, not v4.7 VSP,v4.7,public,laurent,Plane alpha + pixel alpha (on Gen3) [17:13] I think that one has been merged, let me check [17:14] yes it has [17:15] VSP,v4.8,plan,laurent,HGO operation mode selection VSP,v4.8,plan,laurent,HGO support upstream on Gen3 VSP,v4.8,plan,laurent,HGO test application covered by an additional task too, on track it could possibly make it early to v4.7, but no guarantee yet at this point, it will depend on the review process [17:16] VSP,?,prototype,laurent,V4L2 request API upstream VSP,v4.8,public,laurent,V4L2 request API usable prototype the test application I'm using now has already been provided to Renesas [17:17] there's another one in development by Intel provided to whom ? [17:18] they plan to release it but the timing is unclear. it's currently blocked by paper work, and is expected to be released either at the end of this month or at mid-May let me check who I sent it to thanks [17:19] instructions were sent to HARUNOBU KUROKAWA [17:20] CC: Takanari Hayama , TOSHIAKI KOMATSU , Hisao Munakata , Magnus Damm , Ryu Nagasawa , TOMOHIRO KOMAGATA , Tadahiro Takahashi , KOJI MATSUOKA , KOUEI ABE , NAOYA SHIIBA "Re: VSP performance enhancements - Documentation" 11/02/2016 01:22 that uses a modified version of the media-ctl application and a modified version of the yavta application OK, KUROKAWA-san is using this (He is here now :) But not yet working correctly [17:21] (on renesas-driver branch) I've rebased all the patches locally and will resubmit them to renesas-drivers thanks and of course test them with the procedure I've explained in that e-mail I don't expect a new test application to be used at this point [17:22] I'll fix any issue I find kernel side and user space side if needed we'll switch to the new test application when it will be available but there's no urgency for that OK, we would like to test it [17:23] of course [17:24] as soon as it's available I'll try it and will then send instructions thanks that's it on my side. any other question ? when it happen ? the release is expected for end of this month or mid-May [17:25] (14th of May I believe) it depends on the internal paperwork at Intel developers need to ask permission to release tools publicly How about existing test program (?) -> non Intel application ? [17:26] can it be more early ? the program exists already and has been provided, in the e-mail mentioned above [17:27] Yes. as I said, I've rebased the patches, will make sure my test cases work, and submit them to renesas-drivers I don't expect fixes to be needed in user space my target is end of this week for that, weekend at the latest [17:28] Thanks (from KUROKAWA-san :) [17:29] Thanks (from me :) you're welcome apologies for the delay (from me :-)) np any other question ? nothing from me ok I realized I messed up the alphabetical order by the way, I was supposed to go before Niklas [17:30] sorry about that :-) uli___: you're next i thought it was per nick anyway :) DU,?,plan,ulrich,Atomic API test program DU,v4.7,plan,ulrich,HDMI output on Gen3 prototype DU,v4.7,prototype,ulrich,Test setup with HDMI output to HDMI input loopback (without EDID) DU,v4.7,public,ulrich,EDID generation support for the HDMI loopback test setup DU,v4.8,plan,ulrich,HDMI output on Gen3 upstream VIN,v4.7,public,ulrich,Add DV timings support to rcar-vin VIN,v4.7,public,ulrich,Upstream Lager HDMI input bug fixe [17:31] on vin, i have trouble understanding hans's comment: "this just overwrites the adv7180 input with an HDMI input." https://www.spinics.net/lists/linux-media/msg99554.html i just don't know what he is referring to [17:32] I think Hans incorrectly assumes that VIN as both an analog TV input (through ADV7180) and an HDMI input [17:33] can you ask him what he means ? [17:34] just wanted to check if i missed something obvious there can do nothing obvious to me at least [17:35] apart from that, i'm currently working on converting adv7511 and rcar-du to the bridge api for hdmi out on gen3 that has been done already no I agree with pinchartl it looks like he assumes one VIN instance can do both