C/C++: Using Bitfields Effectively

Code Monkey 2 Comments

Introduction

If you’ve ever done embedded development in C/C++, you are probably familiar with bitfields. They are a handy way to reference individual bits in things like hardware registers. The problem is that bitfields can lead to performance problems and race conditions if not used properly. I hope to highlight some of the issues you should consider when using them.

Usage

First, let’s assume you need to check various fields in a hardware register with the following layout:

Bitfield Register Example

You could define the following bitfield to represent this register:

1: struct HwReg
2: {
3:    unsigned int Base : 16;
4:    unsigned int Offset : 8;
5:    unsigned int Rsvd : 5;
6:    unsigned int Flag : 1;
7:    unsigned int Type : 2;
8: };

The total size of this data type is sizeof(unsigned int), with each line defining a different region (field) within that type (this looks confusing when you first look at it). The following code uses the HwReg bitfield to access a memory-mapped register:

1: struct HwReg* pReg = (struct HwReg*)0×80001005;
2:
3: if (pReg->Flag && pReg->Type == TYPE_1)
4: {
5:    void* address = pReg->Base + pReg->Offset;
6: }

Line 1 defines a pointer to the physical hardware register as type HwReg. We can now use this pointer to easily access the register fields. If this isn’t clear, you can read more about bitfields HERE.

Performance Problems

The compiler doesn’t know how to optimize bitfield accesses (especially because the pointers to memory-mapped hardware registers are almost always declared ‘volatile’). This means that every access to a member of the bitfield will require a read of the physical hardware register. This can be orders of magnitude slower than accessing main memory. In the code example above, the hardware register will be read 4 times; once for each field access.

The way to remedy this is to cache a copy of the register value and then operate on that. Consider the following code:

1: unsigned int* pFullReg = (unsigned int*)0×80001005;
2: unsigned int temp = *pFullReg;
3: struct HwReg* pReg = (struct HwReg*)&temp;
4:
5: if (pReg->Flag && pReg->Type == TYPE_1)
6: {
7:    void* address = pReg->Base + pReg->Offset;
8: }

Line 1 defines a pointer to the physical hardware register. Line 2 performs the actual read into a local variable (the slowest part). This local copy is now in main memory and the CPU cache. Line 3 casts the cached value to the bitfield for easy access. Finally, all accesses to the register fields is on the cached value, which can be read very fast from L1 cache.

Another advantage to this approach is when the hardware requires locking before the register can be accessed. By caching the value, you can keep all the locking code localized to a single area of the function. Without caching, you would hold the lock for a longer period of time (possibly forcing other operations to block) and have to make sure to release the lock on every return path (more difficult with exceptions).

NOTE: Remember you are only working with a copy of the register value. If you update a value in the bitfield, you must still copy the updated value back to the register.

Race Conditions

As stated above, each access to a field value generates its own read/write operation. Even if the CPU architecture guarantees that an individual operation is atomic, updating multiple fields are not. Thus, in a multi-threaded application you must lock the entire block of code that operates on the bitfield. I again suggest caching the value, as you only need to lock the actual read/write of the entire register.

Conclusion

Bitfields are a nice language construct that can help make it easier to write clean code (as opposed to using macros and bitmasks). Unfortunately, it’s all too easy to shoot-yourself-in-the-foot with bitfields if you don’t understand the pitfalls. As always, use caution when writing performance-critical code and make sure you understand how to use the available code constructs.

Happy coding!

Share/Save

Motivational Poster: Programming

Code Monkey, Oh So Random 3 Comments

My friend Rob (whom I share an office at work with) made the mistake of sending me the following picture, which I couldn’t resist turning into a motivational poster. I’m just having trouble coming up with the perfect tag line… Comments welcome.

Motivational Poster: Programming

Motivational Poster: Programming

Motivational Poster: Programming

… and my favorite:

Motivational Poster: Office Reality

Share/Save

Inside the Software of the Mars Phoenix Lander

Code Monkey No Comments

Mars Phoenix LanderO’Reilly has a great interview up with NASA’s Peter Gluck, project software engineer for the Mars Phoenix Lander. I always find the design and implementation of mission-critical systems interesting. In short, they’re running a radiation-hardened system (the RAD 6000 board) with a 33MHz CPU, 128 megabytes of RAM, and a PCI peripheral interface… pretty advanced stuff for space. This usually surprises people when they first hear about these systems, but the circumstances require proven technology that is hardened against the perils of outer space (for example, the Hubble Space Telescope was recently upgraded to an Intel 486 processor… the Space Shuttle still runs on hardened PDP-11s).

The software is written in C and running on the VxWorks real-time OS… Lockheed Martin (who wrote the control systems) switched from ADA to C a few years back. There are plenty more interesting details in the article. Here are a few teasers:

The RAD 6000 has built in error detection and corrections. So the hardware does RAM scrubbing. There is a RAM scrubbing that occurs on a continuous basis. And beyond that, we have internal fault protection that monitors the health and safety of the software. And if a software task, for example, fails to respond to a ping, we have pings in the system, then the fault protection task will declare that a fault has occurred and will safe the spacecraft. And what that means, by “safeing”, we mean that the spacecraft will enter into a power and communications safe mode where it will just sit and wait for the ground to respond. It’ll basically phone home and say, I’ve got a problem; somebody tell me what to do.

So if it were to completely lock-up, the hardware has to be stroked every 64 seconds. There’s a watch-stop timer. And so if that 64 second period expires, then the hardware resets and the software is rebooted, and hopefully that clears whatever error occurred. Now in the event that that doesn’t work, we have a whole second set of avionics onboard. So the hardware will try to boot to the same side, and if the same side doesn’t come up and start stroking the watch-stop timer, then it will swap to the other side and boot the first side.

Interviewer: Am I right in assuming that there’s very little process separation in the older RAD 6000 boards?

Peter: Exactly… We have strict coding guidelines that we use. We don’t allow dynamic memory allocation, for example.

These are true fail-safe systems… not the stuff we mortal engineers play with. Click HERE to read the rest of the interview.

Share/Save

Why Securing A Video Stream Is So Difficult

Code Monkey, Tech and Security No Comments

ColorbarsThere was an interesting article this last Friday at the New Scientist about how the contents of encrypted VOIP conversations could still be deduced via traffic analysis. The short version is that many spoken words have a signature to them even when they are encrypted. This signature is related to the size of the data packets used to represent the sound data. Many phonemes in a word have a distinct encoded data size… by analyzing the packet sizes you can deduce the phonemes and thus the spoken word.

This got me thinking I should write about the complex problem of securing a video stream. There are many aspects to securing a video stream: integrity, authenticity, and privacy being the most important. I’m not going to spend time talking about integrity and authenticity, because those are somewhat simpler problems to solve (integritiy = digital signatures, authenticity = digital certificates). The main focus of this post is about privacy; keeping an eavesdropper from deducing the contents of a video stream.

Read the rest…

Share/Save

Create Your Own Motivational Posters

Code Monkey, Oh So Random No Comments

I found this great site MoBuck.com that allows you to easily create your own motivational posters. For $1, you can download a high-resolution version, and for $14 they will send you a 8.5 x 11 inch print. It’s a lot of fun.

Here’s a Mr. T poster I made about not breaking the software build:

Share/Save

Adding reCAPTCHA to Wordpress

Code Monkey 2 Comments

reCAPTCHASo, I finally added a CAPTCHA to my blog to prevent comment spam. I chose to go with reCAPTCHA from Carnegie Mellon University for a couple reasons:

  1. It’s pretty popular right now so I know the code will be actively maintained (important, because CAPTCHAs are broken all the time with bots).
  2. There’s a Wordpress plugin for it, so it was easy to integrate (check out WP-reCAPTCHA).
  3. It offers a benefit to society by helping to digitize old books (read more HERE).

I got the plugin installed, activated, and configured but the CAPTCHA wasn’t appearing in the comments section of my posts… problem. After some searching, I discovered that the Wordpress theme I’m using was missing the following line of code in Comments.php:

<?php do_action(’comment_form’, $post->ID); ?>

This line of code should be placed within the <form> block for submitting comments.

Once I added the above line of code, everything worked. However, my pages wouldn’t validate anymore (arg!). The reCAPTCHA code uses the <style> tag in the body, which can only be used in the HTML <head> block. Here’s the offending line of code in the WP-reCAPTCHA plugin (version 2.8.1, line 300):

<style type=’text/css’>#submit {display:none;}</style>

Since this code simply tries to set the style of the ‘Submit’ button, I just commented it out and everything validated again.

UPDATE 7/22/2008: Paul over a BlogLESS has an addition to this method that fixes a problem with double submit buttons being present. I took his code (HERE) and tweaked it slightly for my theme and added it to my comments.php file (located with my theme files). I replaced:

<input name=”submit” type=”submit” class=”submit1″ id=”submit” tabindex=”5″ value=”Submit Comment” />

with:

<script type=”text/javascript”>
//<![CDATA[
/* Cf. http://www.designlessbetter.com/blogless/posts/making-recaptcha-validate */
document.write('<input name="submit" type="submit" class="submit1" id="submit" tabindex="5" value="Submit Comment" />');
//]]>
</script>

Thanks Paul!

Share/Save

Website validation and embedded YouTube videos

Code Monkey No Comments

W3CSo I took the plunge today and got my website to pass CSS and XHTML validation tests. This ensures that my site renders correctly in most browsers and on mobile devices. Check out UITest.com for links to several validation tools.

The hardest thing to get validated are embedded YouTube videos. The <embed> tag they use is depreciated in XHTML 1.0. As soon as you embed a video, your site will fail validation. I found several code samples for fixing this, but only one of them actually worked. I just had to tweak the size so the output matched that from the YouTube script.

Here’s the HTML:

<object type=”application/x-shockwave-flash” data=”http://www.youtube.com/v/sM_WEohK3bo&hl=en” width=”425″ height=”355″>
<param name=”movie” value=”http://www.youtube.com/v/sM_WEohK3bo&hl=en” />
<param name=”FlashVars” value=”playerMode=embedded” />
<param name=”wmode” value=”transparent” />
</object>

The original script came from HERE.

The only problem I’ve had is with the visual editor in Wordpress auto-replacing this code with the original YouTube code.

Share/Save

Scalability Best Practices: Lessons from eBay

Code Monkey No Comments

eBayInfoQ has a great article up from Randy Shoup, a senior architect at eBay. He discusses the philosophies and practices employed by eBay to ensure their software scales to the demands of the site. I like articles like this because the techniques used by large systems (eBay, Google, Amazon, etc) tend to also apply to smaller systems. These larger systems help flush out the problems that a smaller system may not run into until it is too late.

From the article:

At eBay, one of the primary architectural forces we contend with every day is scalability. It colors and drives every architectural and design decision we make. With hundreds of millions of users worldwide, over two billion page views a day, and petabytes of data in our systems, this is not a choice - it is a necessity.

Click HERE to watch the video presentation and corresponding slides.

Share/Save

Engineer vs. Manager

Code Monkey No Comments

Funny, because it is so true!

A man in a hot air balloon realised he was lost. He reduced altitude and spotted a woman below. He descended a bit more and shouted,
“Excuse me, can you help me? I promised a friend I would meet him an hour ago, but I don’t know where I am.”

The woman below replied,
“You’re in a hot air balloon hovering approximately 30 feet above the ground. You’re between 40 and 41 degrees north latitude and between 59 and 60 degrees west longitude.”

“You must be an engineer,” said the balloonist.

“I am,” replied the woman, “How did you know?”

“Well,” answered the balloonist, “everything you told me is technically correct, but I’ve no idea what to make of your information, and the fact is I’m still lost. Frankly, you’ve not been much help at all. If anything, you’ve delayed my trip.”

The woman below responded, “You must be in Management.”

“I am,” replied the balloonist, “but how did you know?”

“Well,” said the woman, “you don’t know where you are or where you’re going. You have risen to where you are due to a large quantity of hot air. You made a promise, which you’ve no idea how to keep, and you expect people beneath you to solve your problems. The fact is you are in exactly the same position you were in before we met, but now, somehow, it’s my fault.”

Share/Save

Getting Custom Video Streams into WPF Natively

Code Monkey 3 Comments

Over the last 2 weeks, Rob and I have been working hard to get custom video streams into WPF without using WinForms. The issue with WPF is that the application has no access to the lower-level DirectShow stuff to control how the filter graph is built. This leaves the programmer with 2 choices: (1) build a custom WinForms user control and build the filter graph manually (2) register a custom protocol type that will allow DirectShow’s Intelligent Connect to build the graph you want via a specially formed URL.

Option #1 works, but doesn’t allow WPF to do any overlays on the video. This means that all overlays (text, timestamps, images, controls, etc) must be drawn at the DirectShow layer. This is difficult, inflexible, and bypasses all the visual effects power of WPF (not to mention to loose hardware acceleration).

Option #2 is the optimal solution, and was what we set out to prove. This option allows WPF to control the render surface, and thus provide all the overlay functionality the UI designer can expect from WPF. We succeed this week in proving that option #2 is possible, and one of our UI guys went off and used it to create a custom list box for choosing video (good job Brent). The video clips below are live and displayed in real time:

Custom List Box

The videos are displayed using WPF MediaElement objects, and passing in a URL with our custom protocol type and the id of the camera to connect to. The great thing about WPF is that we get all the scaling, reflection, and overlays you see above for free. Another one of our UI guys (Nick) did a demo with live video playing on a 3D cube that can be rotated by the mouse. (UPDATED) Check it out:

Cube Test Application

You can find information about registering custom protocols at http://msdn2.microsoft.com/en-us/library/ms787558.aspx. The DirectShow documentation is at http://msdn2.microsoft.com/en-us/library/ms783323%28VS.85%29.aspx. We downloaded the Windows Vista Platform SDK and built off of the Push Source Filters sample.

Share/Save

« Previous Entries