Introduction
In several video codecs, H264 and HEVC in particular, you may have one or more areas delimited by a region which corresponds to the area of interest that should be displayed.
H264 has the cropping rectangle and it is specified by the amount of pixels in each direction the video should be cropped; nothing outside the region specified should be used according to the spec;
HEVC has the conformance window and the default display window; the former matches the above, the latter specifies a region of interest which can be ignored (or not) depending on the user application;
THEORA has an offset picture defined as width and height and a origin point;
- Other codecs might have some sort of support to the cropping concept, such as VC1, but use is not widespread.
In addition to the various level of cropping, there could be an additional container cropping.
- MOV/MP4, the amount of container cropping is usually 1 on the right or on the bottom of the video, which is used for yuv420p video and odd resolutions;
- MATROSKA on the other hand has a more complete approach and allows cropping on all directions.
Current limitations
The current approach, at least for H264, checks if only container cropping is present or if only bitstream cropping is, and applies it to the decoded frame. This approach mixes lavc and lavf usage and does not take into consideration the (sporadic and unfortunate) case where both container and bitstream cropping are present.
HWAccell testing is impacted by this, since it's not possible to offset the pixel on an opaque surface (which happens on left and top cropping).
There are multiple global flags that interfere with it in some way, such as unaligned or ingorecrop.
Proposed solutions
First of all current behavior should be preserved, possibly with an codec-private option set by default.
Secondly it is proposed that all cropping values should be exported and the application should decide what to do with those, on a full resolution frame.
An AVPacket side data should be added indicating:
- number of rectangles;
- an array containing each rectangle;
- optionally each rectangle is to be marked as bitstream/container;
- an easy to use size with all croppings applied.
An application like avprobe is expected to apply each rectangle separately and report each dimension.
Alternatively it could be possible to specify just a delta on each size, rather than the full size of each rectangle.