FATE, the FATE Automated Testing Environment, is Libav's regression suite. For a user-level view of FATE, including how to use it with and without external samples, please refer to the aforementioned links. To understand the implementation of FATE, and how to add tests, read onward. It is implemented as a series of targets for Make.

Targets

Aside from the usual target, fate, it can be useful to run a subset of tests, such as fate-h264.

A list of targets can be found via make fate-list.

Layout/structure

A good starting point to understand the implementation of FATE is tests/Makefile. This in turn includes a number of files under tests/fate. The first bunch (acodec.mak, vcodec.mak, avformat.mak, and seek.mak) do not require external samples.

A crucial feature of tests/Makefile is that it implements a number of macros, such as ALLYES, XYES, ENCDEC, DEMDEC, etc. These are used as helpers that can evaluate to 'yes' (or 'no'), leading to the primary pattern: someprefix-yes += a_test_or_tests. Here is a real set of examples:

FATE_AVCONV += $(FATE_AVCONV-yes)
FATE-$(CONFIG_AVCONV) += $(FATE_AVCONV)
FATE += $(FATE-yes)

And here is an example from wma.mak, which uses one of the macros rather than a hardcoded yes like above:

FATE_WMA_ENCODE-$(call ENCDEC, WMAV1, ASF) += fate-wmav1-encode

The usual idiom is to build up fate_sometarget-yes with every relevant enabled test. In the above example, FATE_WMA_ENCODE-yes would only have fate-wmav1-encode appended to it if WMAV1 and ASF support are available; otherwise, the ENCDEC macro would not evaluate to yes, so FATE_WMA_ENCODE-yes would be unchanged.

Example file

TODO

Commands, references, comparisons, etc

Test targets can be customized: they can run a particular command, be compared with a specific reference file, etc.

CMD

This is the command to run for the test. Most tests implicitly use avconv; some explicitly set a command.

Examples:

fate-idct8x8: CMD = run libavcodec/dct-test -i

fate-wmapro-2ch: CMD = pcm -i $(TARGET_SAMPLES)/wmapro/Beethovens_9th-1_small.wma

fate-opus-$(1): CMD = avconv -i $(TARGET_SAMPLES)/opus/$(1).mka -f f32le -

CMP

The comparison to use. If it is null, the only thing that matters is whether the command (see CMD) exited normally. Other values include oneline, oneoff and stddev.

FUZZ

FUZZ refers to how much the expected and actual output can differ. It is mostly needed due floating-point rounding being different across architecture or to test otherwise not-bitexact implementations.

REF

This is the path of a reference file with the expected output, a literal string value (such as a hash or short string), or /dev/null.

CategoryInternals