FATE, the FATE Automated Testing Environment, is Libav's regression suite. For a user-level view of FATE, including how to use it with and without external samples, please refer to the aforementioned links. To understand the implementation of FATE, and how to add tests, read onward. It is implemented as a series of targets for Make.
Aside from the usual target, fate, it can be useful to run a subset of tests, such as fate-h264.
A list of targets can be found via make fate-list.
A good starting point to understand the implementation of FATE is tests/Makefile. This in turn includes a number of files under tests/fate. The first bunch (acodec.mak, vcodec.mak, avformat.mak, and seek.mak) do not require external samples.
A crucial feature of tests/Makefile is that it implements a number of macros, such as ALLYES, XYES, ENCDEC, DEMDEC, etc. These are used as helpers that can evaluate to 'yes' (or 'no'), leading to the primary pattern: someprefix-yes += a_test_or_tests. Here is a real set of examples:
FATE_AVCONV += $(FATE_AVCONV-yes) FATE-$(CONFIG_AVCONV) += $(FATE_AVCONV) FATE += $(FATE-yes)
And here is an example from wma.mak, which uses one of the macros rather than a hardcoded yes like above:
FATE_WMA_ENCODE-$(call ENCDEC, WMAV1, ASF) += fate-wmav1-encode
The usual idiom is to build up fate_sometarget-yes with every relevant enabled test. In the above example, FATE_WMA_ENCODE-yes would only have fate-wmav1-encode appended to it if WMAV1 and ASF support are available; otherwise, the ENCDEC macro would not evaluate to yes, so FATE_WMA_ENCODE-yes would be unchanged.
Commands, references, comparisons, etc
Test targets can be customized: they can run a particular command, be compared with a specific reference file, etc.
This is the command to run for the test. Most tests implicitly use avconv; some explicitly set a command.
fate-idct8x8: CMD = run libavcodec/dct-test -i
fate-wmapro-2ch: CMD = pcm -i $(TARGET_SAMPLES)/wmapro/Beethovens_9th-1_small.wma
fate-opus-$(1): CMD = avconv -i $(TARGET_SAMPLES)/opus/$(1).mka -f f32le -
The comparison to use. If it is null, the only thing that matters is whether the command (see CMD) exited normally. Other values include oneline, oneoff and stddev.
FUZZ refers to how much the expected and actual output can differ. It is mostly needed due floating-point rounding being different across architecture or to test otherwise not-bitexact implementations.
This is the path of a reference file with the expected output, a literal string value (such as a hash or short string), or /dev/null.