tdistler.com

It always pays to read the documentation carefully. That’s the moral of this story, but I think the details are still worth reading.

We’ve been using the LGPL build of Ffmpeg for a while now. However, every few months during testing the decoder will segfault. It’s always on some random box, in a release build, in the middle of the night, and nobody can reproduce it (sound typical?). Well, I think I fixed the problem today and I wanted to share what I found.

I ran our decoder through Valgrind while playing an H.264 stream and noticed several errors like the following:

==6757== Invalid read of size 4
==6757== at 0x4D92C50: ff_h264_decode_nal (in/usr/local/pelco/lib/libavcodec.so.52.59.0)


Hum… strange. MPEG-4 doesn’t have this problem.

After triple-checking all our buffer size calculations, memcpy lengths, buffer-overrun checks, etc I was at a loss to explain this. I even added an arbitrary 4 bytes to every allocation but still ran into problems. I finally decided to go line-by-line through our decoder source, cross-referencing with the avcodec documentation.

Found it!

Looking at the documentation for avcodec_decode_video2(...), I found the following note:

Warning:
The input buffer must be FF_INPUT_BUFFER_PADDING_SIZE larger than the actual read bytes because some optimized bitstream readers read 32 or 64 bits at once and could read over the end. The end of the input buffer buf should be set to 0 to ensure that no overreading happens for damaged MPEG streams.

Note:
You might have to align the input buffer avpkt->data. The alignment requirements depend on the CPU: on some CPUs it isn’t necessary at all, on others it won’t work at all if not aligned and on others it will work but it will have an impact on performance.
In practice, avpkt->data should have 4 byte alignment at minimum.

This must have been overlooked when the decoder was written… oops!

Solution

I ended up adding alignment and padding parameters to our buffer objects, which (internally) uses the following methods for allocation:

Windows: _aligned_malloc(), _aligned_free() (requires malloc.h)
Linux: posix_memalign(), free() (requires stdlib.h)

I mention these methods because I think they’re pretty handy and not everyone knows about them. However, don’t forget to read the documentation before using them!