Sed Regular Expression: Find Lines Not Matching A String

Code Monkey 1 Comment

After some digging, I finally found out how to create a regex for Sed (stream editor) that will find a line that does NOT contain a particular string. First, I used ‘find’ to list all the *.cpp files in my source tree:

find . -name “*.cpp” -print

Then I piped the files to ’sed’ via ‘xargs’ (Note: replace the ‘-e’ with ‘-i’ to actually modify the files inline):

find . -name “*.cpp” -print | xargs sed -e ‘/STRING_TO_INGORE/! { d }’

The trick is adding the ‘!’ (exclamation point) after the search expression. Without it, ’sed’ would think you only want lines with the string, not without it.

This is different than another syntax I’ve seen used: /(?!STRING_TO_IGNORE)/.

Here’s another example. Say you want to replace STRING1 with STRING2 only if the first characters of the line (ignoring white space) are NOT “//”… i.e. skip the string replacement in code comments:

sed -i ‘/^[ \t]*\/\/.*/! { s/STRING1/STRING2/ }’

NOTE: ‘[ \t]*’ means ignore 0 or more spaces or tabs.

Find & Replace in Files on Linux

Code Monkey 2 Comments

TuxA lot of solutions I’ve found for recursively replacing text in files is implemented using shell scripts, perl, php, or some other inconvenient way. Rushi got it right by using the Linux command line. Here it is (slightly modified) from his blog:

find . -name “*.cpp” -print | xargs sed -i ’s/[find]/[replace]/g’

where “[find]” and “[replace]” are the things you are searching for and substituting.

To search files with multiple file extensions, use:

find . -name “*.cpp” -o -name “*.h” -o -name “*.c” | xargs sed -i ’s/[find]/[replace]/g’

ADDED 4-13-2009: See comments for other variations.

How To Get An Error Message Describing Why A Library Failed To Load

Code Monkey No Comments

The title’s kind of a misnomer. This post is really to help me remember how to get a human-readable string from a Windows error code… I’m finally tired of always having to look it up :) . However, my current situation revolves around determining why a DLL (or *.so on Linux) failed to load, so that’s why this post it titled the way it is.

I like to disable the annoying default dialog that pops up in Windows when a library fails to load.

SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOGPFAULTERRORBOX | SEM_NOALIGNMENTFAULTEXCEPT | SEM_NOOPENFILEERRORBOX);

Now, here’s the code to get a user friendly text string:

#ifdef WIN32
   LPVOID pStr = 0;
   DWORD_PTR args[1] = { (DWORD_PTR)pFilename };
   FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER |
      FORMAT_MESSAGE_FROM_SYSTEM |
      FORMAT_MESSAGE_ARGUMENT_ARRAY,
      NULL,
      GetLastError(),
      MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
      (LPTSTR)&pStr,
      0,
      (va_list*)args);

   // TODO: Do something with the string pStr here.

   LocalFree(pStr);
#else
   // TODO: Call dlerror() and do something with the string.
#endif

C/C++: Using Bitfields Effectively

Code Monkey 7 Comments

Introduction

If you’ve ever done embedded development in C/C++, you are probably familiar with bitfields. They are a handy way to reference individual bits in things like hardware registers. The problem is that bitfields can lead to performance problems and race conditions if not used properly. I hope to highlight some of the issues you should consider when using them.

Usage

First, let’s assume you need to check various fields in a hardware register with the following layout:

Bitfield Register Example

You could define the following bitfield to represent this register:

1: struct HwReg
2: {
3:    unsigned int Base : 16;
4:    unsigned int Offset : 8;
5:    unsigned int Rsvd : 5;
6:    unsigned int Flag : 1;
7:    unsigned int Type : 2;
8: };

The total size of this data type is sizeof(unsigned int), with each line defining a different region (field) within that type (this looks confusing when you first look at it). The following code uses the HwReg bitfield to access a memory-mapped register:

1: struct HwReg* pReg = (struct HwReg*)0×80001005;
2:
3: if (pReg->Flag && pReg->Type == TYPE_1)
4: {
5:    void* address = pReg->Base + pReg->Offset;
6: }

Line 1 defines a pointer to the physical hardware register as type HwReg. We can now use this pointer to easily access the register fields. If this isn’t clear, you can read more about bitfields HERE.

Performance Problems

The compiler doesn’t know how to optimize bitfield accesses (especially because the pointers to memory-mapped hardware registers are almost always declared ‘volatile’). This means that every access to a member of the bitfield will require a read of the physical hardware register. This can be orders of magnitude slower than accessing main memory. In the code example above, the hardware register will be read 4 times; once for each field access.

The way to remedy this is to cache a copy of the register value and then operate on that. Consider the following code:

1: unsigned int* pFullReg = (unsigned int*)0×80001005;
2: unsigned int temp = *pFullReg;
3: struct HwReg* pReg = (struct HwReg*)&temp;
4:
5: if (pReg->Flag && pReg->Type == TYPE_1)
6: {
7:    void* address = pReg->Base + pReg->Offset;
8: }

Line 1 defines a pointer to the physical hardware register. Line 2 performs the actual read into a local variable (the slowest part). This local copy is now in main memory and the CPU cache. Line 3 casts the cached value to the bitfield for easy access. Finally, all accesses to the register fields is on the cached value, which can be read very fast from L1 cache.

Another advantage to this approach is when the hardware requires locking before the register can be accessed. By caching the value, you can keep all the locking code localized to a single area of the function. Without caching, you would hold the lock for a longer period of time (possibly forcing other operations to block) and have to make sure to release the lock on every return path (more difficult with exceptions).

NOTE: Remember you are only working with a copy of the register value. If you update a value in the bitfield, you must still copy the updated value back to the register.

Race Conditions

As stated above, each access to a field value generates its own read/write operation. Even if the CPU architecture guarantees that an individual operation is atomic, updating multiple fields are not. Thus, in a multi-threaded application you must lock the entire block of code that operates on the bitfield. I again suggest caching the value, as you only need to lock the actual read/write of the entire register.

Conclusion

Bitfields are a nice language construct that can help make it easier to write clean code (as opposed to using macros and bitmasks). Unfortunately, it’s all too easy to shoot-yourself-in-the-foot with bitfields if you don’t understand the pitfalls. As always, use caution when writing performance-critical code and make sure you understand how to use the available code constructs.

Happy coding!

Avoiding Development Disasters

Tech and Security No Comments

Worst Than Failure

I always enjoy reading Alex Papadimoulis’ blog Worst Than Failure (WTF). It recounts stories of software errors, ridiculous coding solutions, and tales of failed engineering interviews. It’s a great place to learn from other people’s mistakes, and have a few laughs along the way.

Occasionally, Alex posts a more serious article about the process of software engineering. I enjoyed his latest on Avoiding Development Disasters.