GPU Accelerated Media Effects in Windows 8 Metro

Windows 8 Metro applications have the ability to add effects to media, such as video playback and live cameras.  This is usually applied to the MediaElement class via the AddVideoEffect (or Audio) call.  There are a few reasons you’d want to make these a/v effects.  One being post-processing of video playback, the other is to be able to apply these effects as they are being encoded and recorded.

These media effects are created by implementing a MediaFoundation Transform.  There is a fine example located here.  Unfortunately that linked sample only shows how to do effects by processing pixels on the CPU.  Since this isn’t the 80’s any more, where we modify pixels in system memory, I wanted to use the GPU and Direct3D and D2D to create GPU accelerated effects for video.

In the sample application I only have it displaying video in a MediaElement with a blur effect applied to it.  But with not too much work I believe you can make this work in a D3D application with IMFMediaEngine and possibly even a WinJS application.  You can also add it to a video capture and record it.

Keep in mind this is just a prototype and not sure it works in every situation (eg, does it need a software fallback?  What about device loss?)

image

Download the application here (You may need the full version of VS11)

And special thanks for Shawn Hargreaves SpriteBatch!

3 comments

  1. gg

    Hey thanks for the code, there’s really not much out on the web about this kind of thing. I’m running into an issue though when I try to build your project on Win8. It’s spitting out all kinds of errors, mainly complaining about missing brackets and semicolons. There’s also an error about InvokedHandlerArgs being undefined on BlankPage.xaml.cs line 72.

    Also maybe you could go into detail how you’re creating the effect. I see you have the VideoXForm.cpp file but how does that use the hlsl shader files that are also included in your project?

  2. Ibrahim Mohamed

    The code doesn’t work on VS11, It outputs too many errors!! And could you give more details on how you’re creating the effect?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s