Here’s a great interview with Microsoft Technical Fellow and author of the CLR garbage collector, Patrick Dussud.
How does GC, work, generally? Why is it important? The GC inside of the CLR is of a specfic type – ephemeral, concurrent (the server version has always been concuurent and now with Background GC on the client in CLR 4, GC is concurrent on the client as well, but there are differences…)
Cyclic Redundancy Checks are a quick and convenient way to verify if a data set has changed. They are not meant to be “secure” like a MD5 or SHA-1, but they offer a very practical solution to most problems. CRC’s are also useful for verifying if 2 data sets are equal. Here’s a simple class I use that’s efficient. Honestly, I don’t remember what flavor of CRC32 it implements (sorry), but it has always worked for what I’ve needed. Cheers!
static const uint32_t kCrc32Table[256] = {
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba,
0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988,
0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec,
0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172,
0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940,
0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116,
0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, Continue reading »
Recently, I was tasked with picking an AAC audio codec library for one of our products. There were several libraries I had to evaluate, and I needed some quantitative metrics for doing the comparison. I’m not what professionals call an “expert listener”, so I had to do the best with what I had. While creating my test plan, I noticed that more people seemed interested in how I was doing testing rather than the actual results. So I decided to share my approach to audio codec testing.
Note: This is intended to be a pragmatic guide for engineers evaluating codecs. It is not a comprehensive treatment of the subject. The goal is to give readers a solid overview and some practical ideas.
Get Familiar with Psychoacoustics
Psychoacoustics is the study of how humans perceive sound. As you might expect, we humans don’t process sound in a perfect, linear fashion. The physical shape of the ear, the transfer function of the Basilar membrane, and the psychological interpretation of the data all affect how we perceive sound (and by extension, how “good” an audio codec sounds to us).
I highly recommend you start by reading this excerpt from Surround Sound: Psychoacoustics Part 1, by Tomlinson Holman (he created THX for Lucasfilm).
Understand the Codec
Make sure you understand the codec you are testing; not necessarily the implementation, but what tools (i.e. methods) the codec uses for compression. Many codecs have different “profiles”, which describe what subset of available tools are used (e.g. AAC). You should also have some idea how each compression tool works and any short-comings it has. This will help guide you in selecting reference audio samples and knowing what artifacts to listen for.
For an introduction to modern audio compression, read Audio Coding: An Introduction to Data Compression Part 1, and Part 2 (discusses MP3 and AAC). I actually suggest buying the book “Introduction to Data Compression”, by Khalid Sayood.
Understand the API
Make sure you actually read the codec documentation and look at any available code samples. This step has more to do with due-diligence than anything, as I haven’t seen a codec API we couldn’t work with, but you need to do this. This will also help you scope the work required to get a working encoder/decoder for future steps (if your lucky, the sample code can be used).
Choose the Reference Audio Samples
An effective test requires multiple audio samples with different characteristics. There are many types of artifacts a codec can introduce, and your choice of audio samples will dictate how easy they are to detect. It’s also important to pick samples that reflect the actual types of sound the codec to have to deal with. For example, if the final system will primarily be encoding speech, then you should choose more speech-oriented references as opposed to music samples.
Some characteristics you might consider:
- Transients (snare drum): Sensitive to pre-echo and noise “smearing”.
- Tonal structure (clarinet, saxophone): Sensitive to noise and “roughness”.
- Natural speech (male and female voices of various languages): Sensitive to distortion and smearing of “attacks”.
- Complex sound (bag pipes): Stresses the codec.
- High bandwidth (bag pipes): Loss of high frequencies and program-modulated high frequency noise.
It is also possible to use synthetic sounds and sweeps, but this is only recommended for the automated objective tests below.
As a basic guideline, you need 10-25 second “raw” samples recorded at the highest sample rate your system needs to work with. It is vital that the samples you choose have never been compressed with a lossy codec (mp3, AAC, etc)… that would severally limit the quality of your test. For sample rate and size, I suggest 48kHz 16-bit PCM, but a lower rate/size makes sense if the final system is limited in this area. It also makes sense to use a sample rate of 44.1kHz, since many quality audio samples can be ripped losslessly from CD. Just keep in mind that the objective PEAQ test mentioned below requires 48kHz 16-bit PCM, so up-sampling may be required.
The audio samples can be stored in whatever container format you want (raw, WAV, etc) as long as your codec test application can unpack it. This is important to keep in mind… you don’t want to accidentally run the WAV header through the codec (yes, I’ve done this). The container format is more of a practical issue, but it was worth mentioning.
Generate Various Test Samples and Observe CPU Load
This step is pretty straight-forward: wrap the codec in an application and encode the reference audio samples at different bit rates. You should choose bit rates that represent the full spectrum of bit rates that will be used in the final system. While you’re encoding, track the CPU usage on the codec and how many cores it’s using. You may even want to do a separate test running many encodes in parallel (this works nicely if the CPU usage is too low to measure accurately). Make sure to consider application overhead and disk I/O when making measurements.
After encoding, you need to decode back to raw PCM. Clearly label your files so you know what bit rate each one was encoded with. These decoded test samples are what we will be comparing to the original reference samples.
Do a Subjective Test
How you conduct your subjective testing will depend on several factors, such as time constraints, cost, and the required test precision. At the low end, you could simply listen to the test samples in a pair of headphones and judge the quality yourself. For a high precision test, you could do a full ITU BS.1116 test using “expert listeners” in a controlled environment. While these examples represent the extremes, there are many permutations that can give you the desired quality of results.
The most common subjective test is called a “double-blind triple-stimulus with hidden reference” test. The listener hears three samples (commonly labeled A, B, and C) for a period of 10 to 25 seconds. A is always the original reference sample. The next two samples, B and C, are randomly assigned either the test sample from the codec or the original reference sample played again (called the “hidden reference”). The listener must then rate the difference between B and A, and C and A, not knowing which one is the test sample. The grading scale is:
- 5.0 Imperceptible
- 4.0 Perceptible, but not annoying
- 3.0 Slightly annoying
- 2.0 Annoying
- 1.0 Very annoying
Ideally, you would conduct several tests and average the results together. If you do the listening test yourself, your results will be limited to your listening skills and understanding of audio codec artifacts. Here’s a summary of factors that affect the quality of your results:
- The quality of the listener.
- The choice of audio samples.
- The number and duration of the testing.
- The testing environment, including speaker/headphone quality, room design, and listener placement.
- The quality of randomization of sample order to remove any correlation between samples.
- Proper statistical analysis of the combined test results.
A proper subjective test is both expensive and time consuming. It’s important to find the right balance for your particular needs.
Do an Objective Test
Evaluating a codec objectively requires testing methods that correlate well to actual human perception. You can’t simply measure the distortion introduced by the codec using traditional measurements like Signal-to-Noise ratio (S/N) and Total-Harmonic-Distortion (THD), because they don’t correlate well to perceived audio quality. Some distortion is imperceptible to the human ear, and codecs take advantage of this to increase the compression ratio.
Fortunately, the ITU has standardized an objective audio test called PEAQ (BS.1387). The acronym stands for Perceptual Evaluation of Audio Quality. PEAQ uses software to model the entire human auditory system (including blood flow noise in the inner ear) to generate a set of metrics that are used to give a final “quality” score. The original reference signal is compared to a signal run through the codec, and the result is a real number between 0.0 and -4.0. The result is interpreted on the following scale:
- 0.0 = Imperceptible
- -1.0 = Perceptible but not annoying
- -2.0 = Slightly annoying
- -3 .0= Annoying
- -4.0 = Very annoying
Obviously, values closer to zero are better.
The test was developed by a similar group of audio experts that developed BS.1116 (mentioned above) and the results have been validated against a long list of subjective tests done using expert listeners.
There are several free and commercial software packages available for doing PEAQ tests. The best free package I’ve found is AFsp from the McGill Telecommunications and Signal Processing Lab. There’s also peaqb, but there’s a comment that it gives incorrect results. AFsp worked great in my tests and included some helpful tools like CompAudio and InfoAudio.
Summary
Hopefully this post has given you a good starting point and some practical ideas for testing audio codecs. My goal was to provide a pragmatic approach with different options depending on what your actual evaluation needs are. This is in no way a comprehensive treatment of the subject; only an overview. I highly suggest reading some of the books I referenced if you’d like a deeper treatment of the subject. Either way, I hope you found this post helpful.
A colleague of mine found a pretty cool profiling tool for Windows called Very Sleepy. It’s simple, straight-forward, free, and it works… everything I like in a tool (this is why I like the Sysinternals stuff so much). It’s also got call-graph and 64-bit support. Definitely worth checking out.
Here’s a nice 4 page reference for Lua 5.1. It was created by Thomas Lauer and can be downloaded from his website. I’ve reposted it here just-in-case… I’ve been bitten before.
I’ve been working lately on embedding Lua 5.1.4 into a C++ application. I’m really impressed with Lua, and most things are pretty straight-forward. However, when it comes to mapping typed enumerations into Lua, I just haven’t found an easy way to do it. For example, let’s say you want to expose a C++ function into Lua called create(), which takes an enumerated type as a parameter. In C++ this would look like:
typedef enum _TYPE {
TYPE_FOO = 0,
TYPE_BAR,
TYPE_BAZ,
TYPE_MAX
} TYPE;
bool create(const TYPE type);To me, it makes the most sense to call create() from Lua with syntax like:
> create(type.foo)
Ideally, we’d also like the following:
- The enumeration is const, and can’t be altered from Lua code.
- The C++ implementation of
create()could validate the type passed to it is the expected type.
Unfortunately, this isn’t as straight-forward as everything else I’ve done with Lua.
Solution (i.e. cut-to-the-chase and explain later)
Okay, I’m a practical guy… download lua_typed_enums.h and include it in your code. Then you can add the enums shown above to Lua by calling add_enum_to_lua():
add_enum_to_lua( L, "type",
"foo", TYPE_FOO,
"bar", TYPE_BAR,
"baz", TYPE_BAZ,
0 );Your enums are now available in Lua as:
type.foo
type.bar
type.baz
Now, your C++ code that takes TYPE as an input and returns a boolean would be written as follows:
static int lua_create( lua_State *L )
{
if( !check_enum_type( L, "type", -1 ) ) {
/* error */
}
int value = get_enum_value( L, -1 );
/* do something cool */
lua_pushboolean( L, 1 );
return (1);
}You can now call your create() method from Lua how we wanted:
> create(type.bar)
Look at the comments in the header file if you want more info about using add_enum_to_lua(), check_enum_type(), and get_enum_value().
How it Works
There’s no direct way to map the expected behavior of C-enums into Lua (const-ness with typing). What I did was to create a read-only Lua table representing the enumerated type and then make each value in that table have a defined type.
Let’s deal with the first task: making a read-only table. This is pretty well documented on the web, but I’ve included a drawing below that would have helped me a lot in the beginning. Let’s assume we want to represent our enum as a Lua table. We could visualize it as follows:
We could still use the type.foo syntax with this table, but a user could assign a new value to the enum like:
> type.foo = 9
To prevent this, we need to deny write access to the table. The trick to doing this is to use a metatable and set the __index and __newindex values.
So how does this work? When the user reads a value from the table (i.e. by using type.foo), Lua fails to find “foo” in the table. So it looks in the metatable for the __index key. If that is set to point to a table, then it will try to lookup the value in that table. In the case above, it finds the “foo” value and returns it like normal. However, if the user attempts to assign to type.foo, Lua follows the __newindex path, which causes a function to get called that does nothing but print an error… effectively, the write is denied. Here’s what happens:
> type.foo = 9
[string "?"]:1: Attempt to modify read-only table
Okay, cool… we’ve made read-only enumerations, but what about typing. Glad you asked! We add one more layer of abstraction, of course <groaning>:
Basically, we make each value a table that includes the actual value and a type (“type” in our example above). The check_enum_type() function looks at the “type” field in the table to make sure the correct type was passed to your function. The get_enum_value() function gets the “value” field… which is equal to the actual C-enum type you’d expect. From the Lua side, you can inspect the enums like:
> print(type.foo.value)
0
> print(type.foo.type)
type
Explicitly, “type” is a read-only table with value “foo”. The value of “foo” is the table { value=0, type="type" }. I hope this explanation helps. Obviously, I’ve been geeking out on Lua a little too much lately 
Yesterday, I needed to get my server’s network information for a Bash script I was writing. Here are the commands so I don’t forget. They were tested using Fedora 12.
Get IP address:
ifconfig eth0 | awk '/inet / {print $2}' | awk -F: '{print $2}‘
Get MAC address:
ifconfig eth0 | awk '/HWaddr/ {print $5}'
Get network gateway IP address:
route -n | awk '/^0.0.0.0/ {print $2}'
When rendering graphics or video to the screen, it’s important to understand the display process; in particular, vertical sync (v-sync). A common problem when starting out is an issue called “tearing”… where the video appears to be torn horizontally down the middle (see picture). I’ve been looking for a good explanation to share about why video tearing occurs and how to solve it (from a technical perspective). I found the following thread over at [H]ard|Forum, which I think does a pretty good job.
For the thread “How VSync works, and why people loathe it“:
There is a technique called triple-buffering that solves this VSync problem. Lets go back to our 50FPS, 75Hz example. Frame 1 is in the frame buffer, and 2/3 of frame 2 are drawn in the back buffer. The refresh happens and frame 1 is grabbed for the first time. The last third of frame 2 are drawn in the back buffer, and the first third of frame 3 is drawn in the second back buffer (hence the term triple-buffering). The refresh happens, frame 1 is grabbed for the second time, and frame 2 is copied into the frame buffer and the first part of frame 3 into the back buffer. The last 2/3 of frame 3 are drawn in the back buffer, the refresh happens, frame 2 is grabbed for the first time, and frame 3 is copied to the frame buffer. The process starts over. This time we still got 2 frames, but in only 3 refresh cycles. That’s 2/3 of the refresh rate, which is 50FPS, exactly what we would have gotten without it. Triple-buffering essentially gives the video card someplace to keep doing work while it waits to transfer the back buffer to the frame buffer, so it doesn’t have to waste time. Unfortunately, triple-buffering isn’t available in every game, and in fact it isn’t too common. It also can cost a little performance to utilize, as it requires extra VRAM for the buffers, and time spent copying all of them around. However, triplebuffered VSync really is the key to the best experience as you eliminate tearing without the downsides of normal VSync (unless you consider the fact that your FPS is capped a downside… which is silly because you can’t see an FPS higher than your refresh anyway).
If the thread is ever unavailable, you can download a PDF version HERE.
After some digging, I finally found out how to create a regex for Sed (stream editor) that will find a line that does NOT contain a particular string. First, I used ‘find’ to list all the *.cpp files in my source tree:
find . -name “*.cpp” -print
Then I piped the files to ‘sed’ via ‘xargs’ (Note: replace the ‘-e’ with ‘-i’ to actually modify the files inline):
find . -name “*.cpp” -print | xargs sed -e ‘/STRING_TO_INGORE/! { d }’
The trick is adding the ‘!’ (exclamation point) after the search expression. Without it, ‘sed’ would think you only want lines with the string, not without it.
This is different than another syntax I’ve seen used: /(?!STRING_TO_IGNORE)/.
Here’s another example. Say you want to replace STRING1 with STRING2 only if the first characters of the line (ignoring white space) are NOT “//”… i.e. skip the string replacement in code comments:
sed -i ‘/^[ \t]*\/\/.*/! { s/STRING1/STRING2/ }’
NOTE: ‘[ \t]*’ means ignore 0 or more spaces or tabs.
A lot of solutions I’ve found for recursively replacing text in files is implemented using shell scripts, perl, php, or some other inconvenient way. Rushi got it right by using the Linux command line. Here it is (slightly modified) from his blog:
find . -name “*.cpp” -print | xargs sed -i ‘s/[find]/[replace]/g’
where “[find]” and “[replace]” are the things you are searching for and substituting.
To search files with multiple file extensions, use:
find . -name “*.cpp” -o -name “*.h” -o -name “*.c” | xargs sed -i ‘s/[find]/[replace]/g’
ADDED 4-13-2009: See comments for other variations.
