Of particular interest to me are the changes to DirectX 10, Media Foundation, and the new DirectX 11. Here are some highlights.

DirectX 11:

• “…resource creation and management has been optimized for multithreaded use, enabling more efficient dynamic texture management for streaming.”
• Several improvements have been made to the high-level shading language (HLSL), such as a limited form of dynamic linkage in shaders to improve specialization complexity, and object-oriented programming constructs like classes and interfaces.”

DirectX 10 improvements:

• “The pipeline also introduces the geometry shader stage, which offloads work entirely from the CPU to the GPU. This new stage enables you to create geometry, stream the data to memory, and render the geometry with no CPU interaction.”
• Predicated rendering performs occlusion culling to reduce the amount of geometry that is rendered. Instancing APIs can dramatically reduce the amount of geometry that needs to be transferred to the GPU by drawing multiple-instances of similar objects. Texture arrays enable the GPU to do texture swapping without CPU intervention.”

Media Foundation improvements:

• “…Media Foundation has been enhanced to provide better format support, including MPEG-4, as well as support for video capture devices and hardware codecs.”
• “In Windows 7, Media Foundation provides extensive format support that includes codecs for H.264 video, MJPEG, and MP3; new sources for MP4, 3GP, MPEG2-TS, and AVI; and new file sinks for MP4, 3GP, and MP3.”
• “In Windows Vista, Media Foundation exposed a relatively low-level set of APIs. These APIs are flexible, but may not be appropriate for performing tasks. Windows 7 adds new high-level APIs that make it simpler to write media applications in C++.”

Option #1 works, but doesn’t allow WPF to do any overlays on the video. This means that all overlays (text, timestamps, images, controls, etc) must be drawn at the DirectShow layer. This is difficult, inflexible, and bypasses all the visual effects power of WPF (not to mention to loose hardware acceleration).

Option #2 is the optimal solution, and was what we set out to prove. This option allows WPF to control the render surface, and thus provide all the overlay functionality the UI designer can expect from WPF. We succeed this week in proving that option #2 is possible, and one of our UI guys went off and used it to create a custom list box for choosing video (good job Brent). The video clips below are live and displayed in real time:

The videos are displayed using WPF MediaElement objects, and passing in a URL with our custom protocol type and the id of the camera to connect to. The great thing about WPF is that we get all the scaling, reflection, and overlays you see above for free. Another one of our UI guys (Nick) did a demo with live video playing on a 3D cube that can be rotated by the mouse. (UPDATED) Check it out:

You can find information about registering custom protocols at http://msdn2.microsoft.com/en-us/library/ms787558.aspx. The DirectShow documentation is at http://msdn2.microsoft.com/en-us/library/ms783323%28VS.85%29.aspx. We downloaded the Windows Vista Platform SDK and built off of the Push Source Filters sample.