FFMpeg provides many powerful features for processing audio and video. One cool thing it can do is resample an audio stream. This allows you to convert, say, a 44.1kHz audio stream down to 8kHz, or up to 48kHz. What’s more, FFMpeg can do the conversion to any arbitrary sample rate. This allows you to do cool things like smoothly changing the audio playback speed over time (see sample code below).
There are many pages describing how to resample audio using the ffmpeg command line application, but what about doing resampling in your own program? To do that, you need to use the avcodec library (libavcodec.so on Linux and avcodec.dll on Windows).
- Include
avcodec.h - Call
avcodec_init()to initialize the FFMpeg library. - Create a resampling context using
av_resample_init()that describes how you want the resampling done. - Call
av_resample()to do the actual resampling on your audio buffer. - When you’re done with the resampling context, delete it with
av_resample_close(). - Finally, link your application against
avcodec,avutil, andzlib(it won’t work on Linux without this one).
Here it is in pseudocode:
#include "libavcodec/avcodec.h"
avcodec_init();
struct AVResampleContext* ctx = av_resample_init( ... );
av_resample( ctx, ... );
av_resample_close( ctx );
That’s it… seriously!
Sample Code (Linux):
Here’s a sample program I wrote that takes a raw 44.1kHz/16bit/mono audio file and plays it back using the pulseaudio API. The catch is that it allows you to specify a “skew” parameter which will cause the audio to dynamically speed up and slow down (via resampling). The amount of resampling is controlled by a sine wave, which is what drives the speed changes.
Download: resample.tar.bz
To unpack and build, type:
$ tar -xjvf resample.tar.bz
$ make
First, run the sample with no skew:
$ ./resample audio_16b_44k_mono_pcm_raw 0
Now, try it with a heavy skew:
$ ./resample audio_16b_44k_mono_pcm_raw -10000
When rendering graphics or video to the screen, it’s important to understand the display process; in particular, vertical sync (v-sync). A common problem when starting out is an issue called “tearing”… where the video appears to be torn horizontally down the middle (see picture). I’ve been looking for a good explanation to share about why video tearing occurs and how to solve it (from a technical perspective). I found the following thread over at [H]ard|Forum, which I think does a pretty good job.
For the thread “How VSync works, and why people loathe it“:
There is a technique called triple-buffering that solves this VSync problem. Lets go back to our 50FPS, 75Hz example. Frame 1 is in the frame buffer, and 2/3 of frame 2 are drawn in the back buffer. The refresh happens and frame 1 is grabbed for the first time. The last third of frame 2 are drawn in the back buffer, and the first third of frame 3 is drawn in the second back buffer (hence the term triple-buffering). The refresh happens, frame 1 is grabbed for the second time, and frame 2 is copied into the frame buffer and the first part of frame 3 into the back buffer. The last 2/3 of frame 3 are drawn in the back buffer, the refresh happens, frame 2 is grabbed for the first time, and frame 3 is copied to the frame buffer. The process starts over. This time we still got 2 frames, but in only 3 refresh cycles. That’s 2/3 of the refresh rate, which is 50FPS, exactly what we would have gotten without it. Triple-buffering essentially gives the video card someplace to keep doing work while it waits to transfer the back buffer to the frame buffer, so it doesn’t have to waste time. Unfortunately, triple-buffering isn’t available in every game, and in fact it isn’t too common. It also can cost a little performance to utilize, as it requires extra VRAM for the buffers, and time spent copying all of them around. However, triplebuffered VSync really is the key to the best experience as you eliminate tearing without the downsides of normal VSync (unless you consider the fact that your FPS is capped a downside… which is silly because you can’t see an FPS higher than your refresh anyway).
If the thread is ever unavailable, you can download a PDF version HERE.
Being a parent now, I found this commercial pretty funny.
I have to hand it to Microsoft… if this is a supposed to be a serious tool (the promo seems to sell it as such) then they should just throw in the towel now before they tarnish their image anymore. The idea behind SongSmith is that you sing the melody and it will auto-generate the backing music… I’m sure you can already see where this is going. Many people have run the lyric tracks from popular songs through it with funny results. I’ll let you find the actual SongSmith demo video on YouTube yourself… I’m posting my favorite SongSmith results below.
(requires Adobe Flash plugin… click HERE to watch it on YouTube)
(requires Adobe Flash plugin… click HERE to watch it on YouTube)
(requires Adobe Flash plugin… click HERE to watch it on YouTube)
My friend Nick brought this video to my attention. It shows how small $100 million dollars is compared to the entire US budget. What really stood out to me was how much of the budget is dominated by welfare handouts… looks like over 80%. I guess we are “all socialists now.”
(requires Adobe Flash plugin… click HERE to watch it on YouTube)
I found my Charlie Hunter CD recently and I’ve been enjoying listening to him again. He’s an amazing musician apart from the fact that he plays the bass and guitar simultaneously. YouTube didn’t exist when I first got into him, and it’s nice now to be able to watch him play. Enjoy!
(requires Adobe Flash plugin… click HERE to watch it on YouTube)
If this video ever gets deleted from YouTube, you can download it HERE.
I’m sorry, but this is too funny not to post…
(requires Adobe Flash plugin… click HERE to watch it on YouTube)
If this video ever gets deleted from YouTube, you can download it HERE.
I finally found the full Marcus Miller concert that includes the song “Blast” which I linked to earlier. Unfortunately, the video has been removed from YouTube. Here it is from FabChannel.
UPDATE 3-17-2009: FabChannel is no longer around 
Click HERE to download a clip from another concert (*.mp4 file playable in QuickTime).
Click HERE to visit Marcus Miller’s website.
I like things that make sense. I’m definitely a pragmatist. Regardless of the principles behind the abortion debate, I’ve always had a problem with the inconsistent definitions of life and death. So I put this question to a friend (and fellow musician) who happens to work for Central California Right To Life.
Josh,
An important issue in the abortion debate is the argument over when life begins; at conception, at birth, or somewhere in between? I’ve heard many opinions over the years, but one (from a fellow engineer) has always struck me as a possible solution. The logic of the argument goes like this: ‘life’ and ‘death’ are inverse states of existence. Therefore, the set of criteria that define ‘life’ and ‘death’ should also be inverse quantities. Thus, we should define ‘life’ by the inverse of the criteria we use for ‘death’, since those criteria are pretty well accepted both legally and morally (of course there are always edge-cases). For example: if someone is pronounced dead when their heart and brain consciousness stop, then life is defined as conscious brain function and a heart beat (over-simplified of course).
This is a very pragmatic argument. It view’s the woman’s body as analogous to a life-support system, thus rendering null the argument that life begins when the baby can exist outside the mother’s womb. This is an important step, because technology renders this a moving target… at some point, an artificial womb may be able to carry an embryo to full term. I’d be interested to get your thoughts on this.
-Tom
Josh responded to my question on his video podcast LifeReport.
Thanks Josh for the response, but you are too gracious with the praise! 
(requires Adobe Flash plugin… click HERE to watch it on YouTube)
