linux-core/nv20_graph.c


1

../shared-core/nv20_graph.c
h1. Multimedia-chat-meeting-2016-10-09

Hello,

We had a multimedia group meeting on 2016-10-09. Here's a summary of the discussions. Please correct any mistake you would notice.

Attendees:

- Kieran
- Laurent
- Morimoto-san
- Niklas
- Ulrich

Magnus was excused due to being in a plane.


Topic 1. Status check for the multimedia tasks
----------------------------------------------

* Kieran

Since last meeting:

- Attended Kernel Recipes in Paris

There were a few multimedia presentations given, in particular about colourspace and DRM.

- Display writeback implementation

This is targeted at Gen3, based on VSP1 writeback. Need a few changes to the VSP driver in the frame end processing to handle writeback and display update races.

For the next two weeks:

- ELC-E
- Display writeback

Issues and Blockers:

- WPF Lockup I was experiencing is likely invalid until I get timings fixed up for a better test.
- Changes to VSP1 timings atomic_flush() must be verified.


* Laurent

Since last meeting:

- Attended Kernel Recipes in Paris

- LVDS mode selection for the DU driver

Patches have been posted, the DT bindings have just been reviewed today. A new 
version will be needed.

- DU support for M3-W

Code tested, works with VGA output. Will post patches after ELC-E.

- DU encoders clean up

Use the drm_bridge framework to handle all external encoders, including the 
VGA DAC with the driver recently submitted by Maxime Ripard. This allows 
deleting the hardcoded list of encoders from the DU driver.

For the next two weeks:

- ELC-E
- New version of the LVDS mode selection patches
- Post M3-W DU support patches
- Continue with the DU encoders clean up
- Post proposal for rotation & histogram API

Issues and Blockers:

None.


* Niklas

Since last meeting:

- VSP1 HGT tests

Extended the tests to cover overlapping hue areas. Patches posted, will be 
merged by Laurent in a separate branch until the corresponding kernel code 
lands upstream.

- VIN Gen3 rework

Make use of the media controller framework instead of V4L2 input selection.

For the next two weeks:

- ELC-E
- VIN support for M3-W proof of concept
- Continue working on media controller-based input selection in VIN
- Discuss ADV7482 upstreaming face to face with Hans

Issues and Blockers:

None.


* Morimoto-san

Since last meeting:

- Handled BSP team multimedia requests
- LinuxCon Europe
- Bunch of audio patches accepted

The other pending patches will be posted after the merge window closes.

For the next two weeks:

- ELC-E
- Meeting with ALSA SoC maintainer at ELC-E

Need to discuss framework historical issues that require cleanup. Due to 
timezone differences progress on that was slow on the mailing list, hope to 
move forward face-to-face.

Issues include

 - old architecture, needs cleanup
 - unbind problem (races, crashes)

Issues and Blockers:

None.

* Ulrich

Since last meeting:

- Reposted Gen2 VIN integration

Updates to the ADV7180 DT bindings are needed to add ports, this won't be 
trivial.

For the next two weeks:

- ELC-E
- Face to face discussions about the ADV7180 DT bindings with Hans Verkuil

Issues and Blockers:

Non-trivial changes to the ADV7180 DT bindings.


Topic 2. BSP patch mining coordination
--------------------------------------

Team leaders have been tasked with mining the BSP for patches and classifying 
them based on the subsystem/device/feature they're related to, with a proposed 
upstreaming plan for each of them. Morimoto-san has already posted a list to 
the Renesas wiki, and Simon got his own list too.

To avoid work duplication, we will use Simon's spreadsheet as the canonical 
list of BSP patches, and update the status as patches are merged in mainline.


Topic 3. BSP team requests
--------------------------

- Cache management on V4L2

The problem is well known but no mainline solution has been developed yet. 
There is however interest in this topic from various companies.

This can be handled as additional task(s), the schedule needs to be discussed.

- Rotation and image partitioning

Image partitioning has been implemented and merged in mainline for v4.9.
Rotation support has been implemented as well but currently blocked on review.
The upstream target is v4.10 at this point.

- V4L2 Request API

The API will be discussed tomorrow during a whole day V4L2 meeting with Hans 
Verkuil and Sakari Ailus among others. More information about the upstreaming 
schedule will be available then.

- VSP1 state bug

Two race conditions have been found recently. One of them has already been 
fixed in mainline (v4.9):

commit bfb4d5be9e1d5a70d0710e815d15a4245eaaafc4
Author: Kieran Bingham <kieran+renesas@bingham.xyz>
Date:   Tue Sep 6 14:07:09 2016 -0300

    [media] v4l: vsp1: Repair race between frame end and qbuf handle

Work is in progress on a second one. Whether the issue found by the BSP team 
is identical isn't known yet. It would be helpful if the BSP could retest with 
the above commit. We can schedule an additional task for this quarter to solve 
the problem if it still occurs.

- DU/VIN DT style difference between ES1.x and ES2.0

The DU and attached VSPs have changed significantly between ES1.x and ES2.0. 
This will require different compatible strings. The "renesas,vsps" property 
will still be used, referencing 3 VSPs instead of 4. There should be no other 
change needed to the DU DT bindings.

For VIN, differences between ES versions are limited to CSI2 routing. This is 
hardcoded in the driver at the moment. As VIN has no IP core version register, 
routing selection has to be done through different compatible strings at the 
minimum. Another option would be to express full routing in DT, but that would 
be more complex and isn't considered as a good solution.

- Is it possible to have VIN on renesas-drivers in 11/M ? (M3/H3)

VIN on Gen3 requires the external HDMI to CSI2 ADV7482 driver. At the moment 
the existing driver is a prototype that hardcodes input selection due to 
missing V4L2 APIs upstream (this topic will be discussed face to face with 
Hans Verkuil this week). Whether the code can be merged in renesas-drivers 
depends if the renesas-drivers tree is a Renesas -next staging area or a BSP 
staging area. We expect the core group to discuss this topic and provide an 
answer.

- Is it possible to have M3 DU on renesas-drivers in 11/M ?

Yes.

- rcar-du + dma-buf + fence

The DU driver supports buffer sharing with dma-buf, but doesn't implement 
fence support. Support for the upstream API can be implemented, but can't be 
tested at this time with the GPU due to the GPU driver not being publicly 
available. We can thus schedule fence support as an additional task, but 
without any guarantee that it will work out of the box with the SGX driver 
stack.

- horizontal lines appears in the plane

The BSP team noticed a display corruption issue with renesas-
drivers-2016-09-20-v4.8-rc7 with the following patches applied:

 - gen3_du_ipmmu.config
 - 0001-linux-v4.8-rc-fcp-get-device-20160901.patch
 - 0002-arm64-dtsi-r8a7795-Enable-IPMMU-node-for-DU0-1-2-3.patch
 - 0003-v4l-vsp1-Add-underrun-hung-up-workaround.patch

If we can get those four patches we will investigate and provide a fix or, if 
the problem is complex, a plan.

- Runtime SRC (Sampling Rate Converter) connection on R-Car sound

The customer would like to use the SRC several times in the audio pipeline. 
This can't easily be handled with the current driver design. The exact use 
case behind the request isn't known, Morimoto-san asked for details. If the 
use case is valid, implementation would require major changes to the driver.


Topic 4. Additional tasks for 12/M
----------------------------------

We drafted the following list of candidates for additional tasks.

* Can be addressed immediately

- GStreamer V4L2 deinterlacer element
- VSP test suite improvements
- VSP image partitioning quality optimization
- Cache management in V4L2
- Fix VSP1 race conditions
- R-Car DU fence support
- Fix Display corruption (horizontal lines) with IPMMU
- Suspend/resume test suite
- Gen3 HDMI output upstreaming

* Needs coordination with Renesas

- VSP suspend/resume support prototype
- Wayland V4L2 compositor performance assessment
- FCP near-lossless compression (FCNL) prototype

* Blocked by hardware availability

- VSP ES2.0 BRS support prototype
- H3 ES2.0 DU support prototype

* Not wanted by Renesas at the moment

- Video codecs support


Topic 5. Next meeting
---------------------

The next meeting will be held on Tuesday 2016-10-25 at 08:00 BST / 09:00 CEST 
/ 10:00 EEST / 14:00 JST.

-- 
Regards,

Laurent Pinchart