I recently (June 2007) did an evaluation of C++ portable runtimes for my employer (a large video security company) for use in our system level code. We were looking for a set of libraries that would abstract the operating system (threads, IPC, File I/O, etc) and allow our code to be portable across many architectures and environments. I’ve decided to publish the results of my evaluation here for anyone interested in portable runtimes.
Spoiler: We chose to go with the ACE version 5.4.1.
I initially considered the following runtimes for evaluation:
- Adaptive Communication Environment (ACE)
- Portable Components (POCO)
- Portable Types (PTypes)
- Boost
- Platinum
- VR Juggler Portable Runtime (VPR)
After considering the criteria we were using for evaluation (see below), I narrowed the field down to 3 candidates:
- ACE
- POCO
- PTypes
I’ve included the selection criteria and evaluation results below. Happy reading.
UPDATE 11/14/2008: Read my post about problems with ACE and debugging memory leaks in Windows.
CANDIDATE SELECTION
Required Features
- Must be written in C++
- Must run on Linux (x86 and PPC) and Windows
- Must have a modular architecture
- Must abstract threading and synchronization
- Must abstract sockets
- Must abstract file I/O
- Must abstract IPC
- Must abstract memory management
Areas That Will Be Evaluated
Listed in order of importance:
- Number of supported platforms
- How actively is it maintained
- Performance
- Total compiled library size
- Future potential
- Ease of use
- Number of relevant features
- Breadth of acceptance
- Documentation
- Level of support
Reasons for Candidate Selection
ACE
| Category | Implementation |
|---|---|
| Threads | ACE_Thread, ACE_Singleton, ACE_Barrier, ACE_Condition, ACE_Guard, ACE_Mutex, ACE_Recursive_Thread_Mutex |
| Sockets | ACE_SOCK_Connector/Acceptor, ACE_SOCK_Stream, ACE_SOCK_Dgram, ACE_ICMP_Socket |
| File I/O | ACE_File, ACE_File_IO, ACE_File_Connector, ACE_OS (C runtime functions… fopen, fseek, etc) |
| IPC | ACE_MEM_Connector/Acceptor, ACE_SPIPE_Connector/Acceptor |
| Memory Mgmt | ACE_Allocator, ACE_Cached_Mem_Pool, ACE_Auto_Ptr |
POCO
| Category | Implementation |
|---|---|
| Threads | Thread, ThreadPool, FastMutex, RWLock, Condition, Event, ScopedLock, ScopedRWLock, Notifications, SingletonHolder |
| Sockets | RawSocket, MulticastSocket, StreamSocket, DatagramSocket |
| File I/O | File, FileInputStream, FileOutputStream |
| IPC | NamedEvent, NamedMutex, Pipe, PipeInputStream, PipeOutputStream, SharedMemory |
| Memory Mgmt | DynamicFactory, MemoryPool, AutoPtr |
PTypes
| Category | Implementation |
|---|---|
| Threads | thread, semaphore, timed semaphore, mutex, rwlock, trigger (condition/event), msgqueue, jobqueue, atomic ops (inc,dec,exchange) |
| Sockets | ipstream, ipstmserver, ipmessage, ipmsgserver, utilities (hostname, iptostring, etc) |
| File I/O | instm, outstm, infile, outfile, logfile (all stream based – defines static pin, pout, perr, pnull for access to stdin, stdout, etc) |
| IPC | namedpipe, npserver, inmemory, outmemory (all stream based) |
| Memory Mgmt | tobjlist (object list – dynamic creation/destruction) |
Reasons for Candidate Dismisal
Boost
- Shared memory support not currently released
- Complex file I/O abstraction
- Only IPC is a pipe file
VPR
- No IPC
- Minimal features
- Geared toward 3D graphics and virtual reality applications
Platinum
- Still in beta
EVALUATION RESULTS
Supported Platforms
| ACE | PTypes | POCO |
|---|---|---|
| Linux (Debian, RedHat, SuSE, Timesys) | Linux | Linux |
| Windows (NT, Embedded NT, 2000, XP, WinCE) | Windows (NT, 2000, XP) | Windows (NT, 2000, XP) |
| Mac OS X | Mac OS X | Mac OS X |
| BSD (FreeBSD, NetBSD, OpenBSD) | BSD (FreeBSD) | BSD |
| UNIX (Solaris, IRIX, HP-UX, Tru64UNIX, AIX, UnixWare, SCO) | UNIX (Solaris) | UNIX (Solaris) |
| OpenVMS | - | OpenVMS |
| QnX Neutrino | - | QnX |
| LynxOS, VxWorks, ChorusOS, RTEMS, OS9, PSoS, MVS OpenEdition, CRAY UNICOS) | Any POSIX-based | Any POSIX-based |
Project Activity
| Type of Activity | ACE | PTypes | POCO |
|---|---|---|---|
| Number of developers “officially” listed | 7 | 3 | 5 |
| Number of contributers | 2000+ | 7 | 10 |
| Year project started | 1993 | 2002 | 2005 |
| Last stable release | Apr 2006 | May 2007 | May 2007 |
| Last release (alpha, beta, or stable) | Apr 2007 | May 2007 | May 2007 |
| Number of downloads per month | ~35000 | 130 (May 2007) | 1494 (May 2007) |
| Number of open bugs (May 2007) | 266 | 4 | 10 |
| Number of bugs fixed (May 2007) | 6 | 0 | 5 |
| Number of forum posts (May 2007) | - | 5 | 18 |
| Number of forum posts w/ replies (May 2007) | - | 4 | 18 |
| Number of news group/mailing list posts (May 31st-June 7th) | 76 | - | 8 |
Performance
Code Review: The information in this table is here to help guage the quality of the implementation details below.
| Metric | ACE | PTypes | POCO |
|---|---|---|---|
| Overall quality | Good | Ok | Good |
| Comments | Good | Ok | Ok |
| Returns values checked | Good | Poor | Good |
| Use of try/catch | Good | Poor | Good |
| Code readability | Poor | Good | Good |
Implementation Details: All of the implementations seem to be fairly compact and efficient. However, ACE has a complex “connector” design model for file I/O that has a lot of indirection. This is probably not much of a performance issue, but it may lead us to use the C-style ACE OS methods (ACE_OS::fopen, ACE_OS::fclose, etc).
| Item | Platform | ACE | PTypes | POCO |
|---|---|---|---|---|
| Mutex | Win32 | Native mutex/critical section depending of scope (process/thread) | Critical section (semaphore for process scope) | Critical section (NamedMutex for process scope) |
| POSIX | pthread mutex | pthread mutex | pthread mutex | |
| Thread | Win32 | CreateThread (or CWinThread if using MFC) | _beginthreadex | CreateThread (can’t set stack size) |
| POSIX | pthread | pthread | pthread | |
| Socket | Win32 | WinSock 2 (WSA methods) | WinSock 2 (Berkley interface) | WinSock 2 (Berkley interface) |
| Other | Berkley sockets | Berkley sockets | Berkley sockets | |
| File I/O | Win32 | Native API (CreateFile, etc) | Native API (CreateFile, etc) | Native API (CreateFile, etc) |
| Other | ::open_f and ::open | ::open | ::open | |
| Condition | Win32 | Native events | Native events | mutexes w/ waitqueue |
| Other | mutexes/semaphores (thread/process scope) | pthread conditions | mutexes w/ waitqueue | |
| Auto pointer | Uses ‘delete’ to remove object | N/A | Uses ref counts via custom interface |
Total Library Size
ACE:
NOTE: ACE has a tool called soreduce that will analyize an application (or set of applications) for it dependencies to ACE, and generate a makefile that builds a custom version of ACE without unused objects.
| Platform | File Name | Release (x86) | Debug (x86) | Release (PPC) | Debug (PPC) |
|---|---|---|---|---|---|
| Linux | libACE.so | 1.7 MB | 8.1 MB | 2.0 MB | 8.4 MB |
| Windows | ACE.dll | 1.1 MB | 2.3 MB | - | - |
| Windows | ACE.lib | 1.9 MB | 2.9 MB | - | - |
PTypes:
| Platform | File Name | Release (x86) | Debug (x86) | Release (PPC) | Debug (PPC) |
|---|---|---|---|---|---|
| Linux | libptypes.so | 0.20 MB | 0.20 MB | 0.23 MB | 0.22 MB |
| Windows | ptypes21.dll | 0.16 MB | 0.23 MB | - | - |
| Windows | ptypes21.lib | 0.27 MB | 0.27 MB | - | - |
POCO:
| Platform | File Name | Release (x86) | Debug (x86) | Release (PPC) | Debug (PPC) |
|---|---|---|---|---|---|
| Linux | libPocoFoundation.so | 1.3 MB | 8.0 MB | 1.4 MB | 7.6 MB |
| Linux | libPocoNet.so | 0.79 MB | 5.7 MB | 0.85 MB | 5.2 MB |
| Linux | libPocoUtil.so | 0.24 MB | 2.0 MB | 0.26 MB | 2.0 MB |
| Linux | libPocoXML.so | 0.53 MB | 3.5 MB | 0.59 MB | 3.3 MB |
| Windows | PocoFoundation.dll | 1.0 MB | 2.2 MB | - | - |
| Windows | PocoFoundation.lib | 1.1 MB | 1.1 MB | - | - |
| Windows | PocoNet.dll | 0.60 MB | 1.3 MB | - | - |
| Windows | PocoNet.lib | 0.72 MB | 0.72 MB | - | - |
| Windows | PocoUtil.dll | 0.26 MB | 0.62 MB | - | - |
| Windows | PocoUtil.lib | 0.25 MB | 0.25 MB | - | - |
| Windows | PocoXML.dll | 0.50 MB | 0.98 MB | - | - |
| Windows | PocoXML.lib | 0.50 MB | 0.50 MB | - | - |
Future Potential
ACE:
- Started in 1993.
- Still under active development.
- I received 2 responses to my newsgroup post within 30 minutes.
- I emailed Douglas Schmidt (creator of ACE) directly with some questions, and received a reply within 45 minutes.
- Has very wide acceptance.
- The next stable release is scheduled for the end of summer 2007.
- Several companies commerically supporting it.
PTypes:
- Started in 2002.
- The creator took a 2 year break from working on it, but is now back.
- Has a smaller following than ACE.
- Has minimal activity on it’s forum.
- Couldn’t find any commercial support for it.
POCO:
- Started in 2005.
- Still under active development.
- Has a small but growing following.
- Light activity on the forum, but questions answered quickly.
- Commerical support provided by the same company that created it.
Ease of Use
NOTE: These results are very subjective, as no formal methods were used.
POCO provided that most straight-forward set of classes out of all the candidates evaluated. The functionality of the classes and associated methods should be obvious to any software engineer. PTypes was a close second behind POCO regarding ease-of-use. ACE has a high learning curve, with many architectural design concepts that may be new to some. However, once up the learning curve, ACE doesn’t seem much harder to use than the other candidates (except for the shear volume of classes available).
Relevant Features
ACE is by-far the most complete feature set of all the candidates. However, it is unlikely we will utilize many of the features of ACE, but it does provide us the option to so if we need. PTypes seems a little light on some of the core features, so we might have to still create some classes to fully meet our needs. POCO is complete, providing a clean implementation and architecture. The nice thing about POCO is it’s feature set can easily be limited to only an OS abstraction layer. ACE has so many other features, that it might have the tendency to creap outside the domain of simple OS abstraction.
I’ve avoided listing all the relevant features here, because the documentation for all candidates is freely available online.
Breadth of Acceptance
| ACE | PTypes | POCO |
|---|---|---|
| ATD, Airforce Research Lab, BAE Systems, BBN, Boeing, CDI/GDIS, Cisco, Comverse, DARPA, Ericsson, Experian, Global MAINTECH, Hughes Network Systems, IBM, Kodak, Krones, Lockheed Martin, Lucent, Microsoft, Mitre, Motorola CGISS, Motorola Iridium, OCI, Office of Naval Research, Oresis, OSC, OTI, Nokia, Nortel, NSF, PrismTechnologies, QNX, Qualcomm, Raytheon, Riverace, SAIC, Siemens, Sprint, Telcordia, USENIX, Veritas | - | TAC AB, RF-iT |
Documentation
| Category | ACE | PTypes | POCO |
|---|---|---|---|
| Documentation quality | Good | Good | Good |
| Tutorials and Examples/Quality | Yes (good) | Yes (minimal) | Yes (good) |
| Books on the project | Yes (3) | No | No |
Level of Support
| Category | ACE | PTypes | POCO |
|---|---|---|---|
| Forum | No | Yes | Yes |
| Mailing list/Newsgroup | yes | No | Yes |
| Level of forum/newsgroup/list activity | High | Minimal | Some |
| Companies providing commercial support | Riverace, Remedy IT, Terabit, OCI | - | Applied Informatics (POCO creators) |
