FFMpeg provides many powerful features for processing audio and video. One cool thing it can do is resample an audio stream. This allows you to convert, say, a 44.1kHz audio stream down to 8kHz, or up to 48kHz. What’s more, FFMpeg can do the conversion to any arbitrary sample rate. This allows you to do cool things like smoothly changing the audio playback speed over time (see sample code below).
There are many pages describing how to resample audio using the ffmpeg command line application, but what about doing resampling in your own program? To do that, you need to use the avcodec library (libavcodec.so on Linux and avcodec.dll on Windows).
- Include
avcodec.h - Call
avcodec_init()to initialize the FFMpeg library. - Create a resampling context using
av_resample_init()that describes how you want the resampling done. - Call
av_resample()to do the actual resampling on your audio buffer. - When you’re done with the resampling context, delete it with
av_resample_close(). - Finally, link your application against
avcodec,avutil, andzlib(it won’t work on Linux without this one).
Here it is in pseudocode:
#include "libavcodec/avcodec.h"
avcodec_init();
struct AVResampleContext* ctx = av_resample_init( ... );
av_resample( ctx, ... );
av_resample_close( ctx );
That’s it… seriously!
Sample Code (Linux):
Here’s a sample program I wrote that takes a raw 44.1kHz/16bit/mono audio file and plays it back using the pulseaudio API. The catch is that it allows you to specify a “skew” parameter which will cause the audio to dynamically speed up and slow down (via resampling). The amount of resampling is controlled by a sine wave, which is what drives the speed changes.
Download: resample.tar.bz
To unpack and build, type:
$ tar -xjvf resample.tar.bz
$ make
First, run the sample with no skew:
$ ./resample audio_16b_44k_mono_pcm_raw 0
Now, try it with a heavy skew:
$ ./resample audio_16b_44k_mono_pcm_raw -10000
What Is It?
ThreadNinja is a Linux library my team created that tracks pthread_create() and pthread_join() calls in an application. It prints a stacktrace where each thread is created and where it is joined. Any rogue (unjoined) threads are reported when the application exits. ThreadNinja is unobtrusive: it does NOT have to be compiled into the code. This means you can use it on applications you didn’t compile.
We found it useful and thought we’d share it. It’s be no means production code… just a tool. Hack on it, expand it, change it… whatever. It’s pretty small, so it should be easy to dive right in. We’ve released it under the BSD license.
Cut To The Chase
You can checkout the source code from Google Code, or download the version 1.0 tarball directly (threadninja.tar.gz).
To build ThreadNinja, simply untar it and call make:
> tar -zxf threadninja.tar.gz
> make
Now, simply use LD_PRELOAD to run the application:
> LD_PRELOAD=/path/to/threadninja/build/libthreadninja.so.1 TheApplication
If you don’t see function names in the stacktraces that are generated, then the application needs to be compiled with debug symbols. For my test app, I had to compile with the -rdynamic option:
> g++ -Wall -rdynamic main.cpp -lpthread
This causes the global symbol table to be included in the executable, which contains all the application’s function names. For more info, look at the --export-dynamic option on the GNU linker (ld) man page.
The Story Behind ThreadNinja
My team was assigned to stabilize a large video application that runs as a Linux-based appliance. The application consisted of 100,000+ of lines to code that was a tangle of build warnings, circular references, and many creative hacks. Our particular task was to fix a persistent set of seg-faults and memory leaks.
Continue reading »
High-performance timing is hard… no doubt about it. I can’t tell you how many times I’ve seen high-performance timing code done wrong. Timing is one of those things where a little knowledge can be problematic; the code may work, but it either won’t perform or will exhibit “unexplained” behavior. The purpose of this post is to explain a foundational component to getting timing right: the clock. I won’t focus on theory… this post is meant to be pragmatic.
Note: I’m talking here about interval timing (i.e. accurately measuring the duration between 2 events). This is different than synchronizing different clocks or maintaining accurate wall time.
Anatomy of the Clock
The first mistake most people make when doing timing is to use functions like gettimeofday(), GetSystemTime(), etc. These functions return what is called “wall time”… time that corresponds to a calendar date/time. These clocks suffer from the follow limitations:
- They have a low resolution: “High-performance” timing, by my definition, requires clock resolutions into the microseconds or better.
- They can jump forwards and backwards in time: Computer clocks all tick at slightly different rates, which causes the time to drift. Most systems have NTP enabled which periodically adjusts the system clock to keep them in sync with “actual” time. The adjustment can cause the clock to suddenly jump forward (artificially inflating your timing numbers) or jump backwards (causing your timing calculations to go negative or hugely positive).
For interval timing, all that’s needed for a clock is a simple counter that increments at a stable rate. For high-performance timing, the rate this counter increments should be high. A related constraint is that the counter must be monotonic (can never “tick” backwards… ever). The counter may overflow and wrap back to 0, but using unsigned math in your timing calculations can compensate for that (see example below).
Something to note: since we are usually measuring short durations, the drift of the clock is so small that we aren’t concerned by it (what matters is the drift between successive reads, not total drift over time).
Continue reading »
We ran into a weird problem the other day where our Linux video display appliance would lose audio support when the process was restarted. The audio was supposed to play through a custom joystick-keyboard that was attached via USB (the keyboard is used by security guards to PTZ cameras, control monitors, etc). The audio could be heard just fine when the box first booted, but if the application restarted audio would be lost.
Looking at the logs, we found that our audio pipeline was failing to open /dev/dsp on the restart. We then used lsof to list the open file descriptors to see which process currently held /dev/dsp:
# lsof | grep /dev/dsp
ntpd 18857 root 16u CHR 14,3 180099 /dev/dsp
What!?!?… why the heck is NTP opening the sound device and how did it steal it from us??? After some discussion we started remembering a problem in the past with ntpd stealing our SNMP diagnostics port. This just didn’t make any sense.
Digging into our appliance code, we found this line:
system( "service ntpd restart" );
This would be called each time we were notified by the security system that the NTP server address had changed (which fired once each time the process was started so we could get the initial address). But this still didn’t explain why NTP took over ownership of our file descriptors on restart.
Long story short: system() is implemented as fork() followed by execv(). By default, fork() gives a copy of the parent’s file descriptors to the child process (i.e. the ntpd child process got a copy of the /dev/dsp file descriptor). To prevent this, you have to set the FD_CLOEXEC flag on the file desciptors you don’t want copied.
For example:
fd = open( "/dev/dsp", O_RDWR );
fcntl( fd, F_SETFD, FD_CLOEXEC );
Conclusion: setting the FD_CLOEXEC flag on the /dev/dsp file descriptor fixed the problem for audio. However, most of the other file desciptors still got owned by ntpd. Did we go back and set the FD_CLOEXEC flag on all file descriptors, you ask? Nope. It turns out we had a script monitoring the NTP config file and restarting ntpd for us when the file got updated… we just had to update the config file and remove the system( "service ntpd restart" ) call.
Oh, and the reason audio worked on first boot but not subsequent restarts was due to a weird race condition around when /dev/dsp got opened.
After some digging, I finally found out how to create a regex for Sed (stream editor) that will find a line that does NOT contain a particular string. First, I used ‘find’ to list all the *.cpp files in my source tree:
find . -name “*.cpp” -print
Then I piped the files to ’sed’ via ‘xargs’ (Note: replace the ‘-e’ with ‘-i’ to actually modify the files inline):
find . -name “*.cpp” -print | xargs sed -e ‘/STRING_TO_INGORE/! { d }’
The trick is adding the ‘!’ (exclamation point) after the search expression. Without it, ’sed’ would think you only want lines with the string, not without it.
This is different than another syntax I’ve seen used: /(?!STRING_TO_IGNORE)/.
Here’s another example. Say you want to replace STRING1 with STRING2 only if the first characters of the line (ignoring white space) are NOT “//”… i.e. skip the string replacement in code comments:
sed -i ‘/^[ \t]*\/\/.*/! { s/STRING1/STRING2/ }’
NOTE: ‘[ \t]*’ means ignore 0 or more spaces or tabs.
A lot of solutions I’ve found for recursively replacing text in files is implemented using shell scripts, perl, php, or some other inconvenient way. Rushi got it right by using the Linux command line. Here it is (slightly modified) from his blog:
find . -name “*.cpp” -print | xargs sed -i ’s/[find]/[replace]/g’
where “[find]” and “[replace]” are the things you are searching for and substituting.
To search files with multiple file extensions, use:
find . -name “*.cpp” -o -name “*.h” -o -name “*.c” | xargs sed -i ’s/[find]/[replace]/g’
ADDED 4-13-2009: See comments for other variations.
Ok, some posts are clearly just to help me remember how to do things… this is one of them. The Subversion source control system keeps private information in .svn directories. There is one such directory for EVERY directory in your source tree. Here’s how you recursively delete ALL the .svn directories from the current directory in Linux (or Cygwin in Windows).
rm -rf `find . -type d -name .svn`
NOTE: Those are back ticks around the ‘find’ command, not apostrophes. I recommend you just run the ‘find’ command first and verify it is listing the directories you expect.
So here’s a cool feature of GNU’s implementation of libc: you can get a stack backtrace (as an array of strings) dynamically in your code. This can be really useful when trying to determine the code path taken when an error occurs. Most times, it’s faster to just run the code in a debugger and use it to display a backtrace, but there are instances when doing it programmatically is your best option. For example, you could get a backtrace in your application’s exception handler and use it to augment error log messages.
First, you need to include execinfo.h to your code:
#include <execinfo.h>
Next, call the backtrace() function to get an array of void pointers that represents the current stack (the pointers are the return addresses for each stack frame).
void* tracePtrs[100];
int count = backtrace( tracePtrs, 100 );
The backtrace() function returns the number of entries in the array (read the man pages for more info about the array size).
Finally, you need to resolve the function names associated with the pointers. You have 2 options: backtrace_symbols() and backtrace_symbols_fd(). Both of these methods resolve the pointers to strings, but the difference is that backtrace_symbols() allocates the strings on the heap while backtrace_symbols_fd() writes the strings to a file descriptor that you can read. Just keep in mind that backtrace_symbols() won’t work if the heap has been trashed.
Here’s an example using backtrace_symbols():
char** funcNames = backtrace_symbols( tracePtrs, count );
// Print the stack trace
for( int ii = 0; ii < count; ii++ )
printf( “%s\n”, funcNames[ii] );
// Free the string pointers
free( funcNames );
NOTE: Make sure you call free() on the array of strings returned from backtrace_symbols().
For more information, here’s a good article from the Linux Journal.
I saw a Linux logo I liked on Google Images a while back, but all instances of it have been removed. I got tired of searching, so last night I hacked around in Photoshop and recreated it.
Debugging C++ templates is difficult. Debugging C++ templates with GDB can be an act of torture for even seasoned GDB users. I like GDB, but there are some tricks you should know when using it to debug templates. In this post, I deal with setting breakpoints.
Breakpoint Basics:
Setting a breakpoint in GDB is supposed to be simple. Here we set a breakpoint at line 50 in file main.cpp:
(gdb) b main.cpp:50
Breakpoint 1 at 0×804937a: file main.cpp, line 50.
We can also use the function name and GDB will attempt to find the correct location for us:
(gdb) b DoSomething
Breakpoint 2 at 0×8049334: file main.cpp, line 150
Simple, right? Just wait…
Breakpoint Gotchas:
GDB’s breakpoint logic is pretty handy for simple projects, but it can break down fast when things get more complicated.
For example, let’s say your application is plugin-driven, with each plugin being a separate library. Now assume each plugin has a Plugin.cpp file under it’s own Source directory. Try to set a breakpoint in the Initialize() method of the Plugin class:
(gdb) b Initialize
Breakpoint 3 at 0×8049717: file main.cpp, line 230
Oops! There is an Initialize() method in main.cpp and GDB thought that’s where we wanted to put it: wrong!
