Migrating to Libav11
Libav 11 aims to be fully API compatible with Libav 10, so no source level modification is required to you own project.
Since however it is not ABI-compatible, you do need to recompile your software against the new release.
It is advisable to update your code in preparation for Libav 12.
The Active Format Descriptor for MPEG2 is a frame-level variable, so it has been moved from AVCodecContext.dtg_active_format to a proper frame side data, AV_FRAME_DATA_AFD.
All values will have to be accessed with a proper AV_ rather than FF_ prefix.
Unused AVCodec options
The following defines and FF_IDCT_IPP, FF_DEBUG_PTS, FF_CODER_TYPE_DEFLATE, FF_DCT_INT the following options ipp, pts, deflated, int, scenechange_factor were unused and have been deprecated.
Private Codec Flag options
The following defines CODEC_FLAG_MV0 and CODEC_FLAG_NORMALIZE_AQP are replaced with "mv0" and "naq" in the "mpv_flags" private option of the mpegvideo encoders.
Similarly CODEC_FLAG_GMC is replaced by the "gmc" private option of the libxvid encoder.
Finally CODEC_FLAG_INPUT_PRESERVED is deprecated. Its functionality is replaced by passing reference-counted frames to encoders.
When using a custom get_buffer() callback, the previous API required the caller to allocate frames with a layer of padding on all sides, unless the caller set the CODEC_FLAG_EMU_EDGE flag (then so-called "emulated edges" were used). Not using emulated edges was thought to be more efficient.
However, actual performance tests revealed that emulated edges are just as fast (or, in some cases, faster) than real edges. Since real edges make the public API more complicated, they were deprecated.
In the current code, all decoders use emulated edges and do not require and kind of padding actually present around frames (i.e. as if the CODEC_FLAG_EMU_EDGE flag was always set). Therefore, all callers setting this flag can simply drop it.
Previously, to provide a hint to the muxer about the desired stream timebase, it was necessary to set the timebase not on the corresponding AVStream, but on its codec member. This was found to be confusing to the users and incompatible with future simplifications. The new way is directly setting the AVStream timebase.
The old pixel format enum values (PIX_FMT_*, defined in pixfmt.h) are replaced with properly namespaced versions (AV_PIX_FMT_*).
The old pixel format flags (also PIX_FMT_*, but defined in pixdesc.h) are replaced with properly namespaced and more easily distinguishable AV_PIX_FMT_FLAG_*.
The array of the pixel format descriptors was previously exported directly. This prevented expanding the descriptor struct in an ABI-compatible way and on some platforms there are various problems with exporting data symbols. To solve those two problems, the array is no longer exported. Instead, av_pix_fmt_desc_get() and av_pix_fmt_desc_next() should be used to get a pixel format descriptor or iterate over all pixel format descriptors respectively.
Some headers (aes.h, md5.h, sha.h and tree.h) used to export the size of the context structure, that would be allocated by the user. Since there are various problems associated with exporting data symbols on some platforms, those were replaced by constructor functions (av_*_alloc()).
The header audioconvert.h, which contained exclusively channel layouts-related code was renamed into the more appropriate channel_layout.h.