This page is documenting a work in progress feature do not assume it is available on the mainline branch.
MVC is an extension documented in the H.264 standard, specifically described in Annex H. It describes how to encode multiple views sharing the same temporal reference.
It is used in the Blu-Ray specification, in a DVB standard and in various research projects.
Each view requires a separate decoder context and each decoder must take in account the fact the reference list could reference frames from different views.
Additional NAL Units
Three new NAL Units are introduced: PREFIX, SUBSET_SPS, EXTENDED_SLICE.
- Extend the immediately next NAL adding view-specific information. Only NAL videos are affected.
- Replaces the normal SPS for the specific view
Video SLICE for any layer besides the first, to be decoded in an extended AVFrame
AVFrame should have AV_FRAME_MULTIVIEW set and priv should contain a pointer to all extra frames.
Only after parsing SUBSET_SPS the number of views is known, if the number changes a reinit/reallocation is needed. To preserve compatibility with existing h264 decoder, new contextes should be allocated on demand.
The NAL Parser should split each EXTENDED_SLICE since they could refer to different views and not skip NALs > 13.
PREFIX and EXTENDED_SLICE NALUs carry an header extension of three additional bytes.
The slice threading should be disabled at least initially. Frame threading should work correctly as long the contextes get propagated correctly.
The SUBSET_SPS interaction is described in H.7.4.2. It is required to keep a separate list of SPS for this specific kind (shared across all the views) and use it only when decoding EXTENDED_SLICE.
The separate getter and setter must be introduce to abstract the interaction.
VOIdx <-> ViewId matching
According H.126.96.36.199.1 and JM, we have two identifiers one for the extended slice and one for the sps. The match-list depends on the current active sps.
Slice multithreading is not possible when decoding MVC, but the structure of context initialization for different layers is very similar.
In slice threading a global reference to the SPS is kept across different contexts. I have to do a reinit when dimensions are wrong, but this has to be done only for the first context, all the other will reuse that reference, or error out.
So when you decode the slice header you have to consider using the same context twice, and not two different ones. We should split the decode_slice_header function in two, one for normal and slice threading, the other for mvc decoding.
Use alerts to point to other blueprints
// code snippets here ...
How it is implemented internally. Put the information useful to implement it or extend it here.