Windows 8 Metro XAML UI and Direct3D 11 Interop

One feature of WPF that I always liked was the D3DImage class in WPF.  It allowed you to render any Direct3D surface to WPF airspace.  This came in handy for when the retained mode XAML system (or WPF 3D) was not not flexible enough.  This opened up a wide range of opportunities in the user interface, such as hardware accelerated video, and high quality 3D graphics.  Windows Phone 7 also has a similar feature, where in Silverlight you can mix XNA and XAML in the same user interface.

At the time of this writing, the new Windows 8 Metro XAML user interface, (which is prerelease) that feature is missing at the moment.  You can make a Direct3D11 Metro application or a XAML application.  No mixing allowed.  As an intermediate solution I wanted to take a crack on what can be done about this.  Here’s a quick video of the result, with apologies for my crappy camera.

How does this work?

Before I get anyone too excited, let me just say it involves a GPU read-back to system memory.  Yeah, I know it’s not optimal, but it’s better than nothing :).  Now that we got that out of the way…

If you aren’t having hurdles to jump over, then you aren’t using pre-beta software.

I was going to be writing this in C++ for a number of reasons, so I spent a good amount of time reading the MSDN documents.  I was happy to see there was a WriteableBitmap class as at least this will let us render pixels.  My excitement dwindled when I found the WriteableBitmap::PixelBuffer property returned an IBuffer.  Why does this stink?  IBuffer only has two properties, “Capacity” and “Length”.  There doesn’t seem to be a “GetBuffer” method.  After some digging I found some examples of how to do this in C# by using some WinRT extension methods.  This was little help because I was doing this in C++.  I decided I needed a little desperation, so I pulled out reflector to see how .NET was getting the pixel buffer.  Low and behold, there is an undocumented interface!   Here is the COM interface I rewrote in native:

image

Ready for pixels!

Now that we had a way to update the WriteableBitmap we needed pixels.  With my sheer amount of weekend laziness, I decided to just hack up one of the Direct3D11 Metro samples.  To make one of these samples work, I only had to do a minimal amount of changes.  Most notably moving all the code to  WinRT DLL, removing all swap chain code and replacing it with an ID3D11Texture2D.  I had a few other pain-points with “Content” in WinRT DLLs, but I figure it’s my ignorance or pre-beta blues.

Performance?

Like I said before, we’re downloading the texture from video memory to system memory, just to be copied and sent back to the GPU…so there is a performance hit.  In my testing I’ve seen around 40 – 55 FPS.  We can probably increase performance a little more with maybe a few more tricks. 

Codes.  Shut up and gimme codes!

You can download the code example here.

Included is a reusable D3DRenderer control that you can use from C++ or C# and sample application.

Have fun…and I assume no liability.

7 comments

  1. Pingback: Dew Drop – October 11, 2011 | Alvin Ashcraft's Morning Dew
  2. Trackback: Microsoft Weblogs
  3. luke

    When it was leaked through that Win8 will have a new XAML based UI framework, based on Direct2D and DX11, I was really excited because I thought that this time, they would get the interop stuff right. If the framework is based on D2D/DX11 then it must finally be easy to interop with native DX stuff. Finally we would get fluid D3D overlays and fluid video in our XAML UIs. Without intermediate layers and memory copying and sync issues.

    It is very disappointing to hear that this is currently not supported. Hopefully they will come up with a solution until beta or release version. A software buffer based writable bitmap will never come close to having access to a “real” D3D surface in GPU memory. Both perfomance wise, as well as from the fluidity and sync behavior.

  4. Pingback: Jeremiah Morrill & The Scholarly Kitchen « Kynosarges Weblog
  5. Pingback: Grand Subscriptions & Miscellaneous Update « Kynosarges

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s