Pixel Shaders for Xaml Windows Runtime

Pixel Shaders first popped up in WPF around .NET 3.5 and were a replacement for the very slow BitmapEffects. This allowed developers to make GPU accelerated shader effects, such as Blur and DropShadow. When Silverlight started picking up steam, they were also added, but as a CPU (concurrent) rasterized version. Since then pixel shader effects have all but disappeared from XAML frameworks such as Windows Phone or XAML in Metro applications.

Windows 8.1 brings a lot of new features that can now facilitate pixel shaders of Xaml UIElements and is the goal of this library. Before any one gets too excited, there are limitations. Most imposed by the XAML framework and some by my library. But first, let me explain further.

How does it work?

The most trivial explanation on how this is accomplished: We use the new RenderTargetBitmap to rasterize a UIElement. The result is then loaded to a GPU surface and Direct2D is used to apply a pixel shader. The shaded surface is then presented with SurfaceImageSource. The original UIElement is left at 0.0 opacity, so it still remains interactive and makes it appear as if the shaded result is interactive.

To give a better perspective, here’s a sample of the usage:

<xamlFx:ShaderContentControl EffectRenderStrategy=”RenderingEvent”

HorizontalAlignment=”Left”

HorizontalContentAlignment=”Stretch”

VerticalAlignment=”Stretch”>

<xamlFx:ShaderContentControl.Effect>

<xamlFx:BlurEffect Radius=”{Binding Value, ElementName=amountSlider}” />

</xamlFx:ShaderContentControl.Effect>

<Button Content=”BUTTON”

Width=”300″

Height=”300″

BorderThickness=”4″>

</Button>

</xamlFx:ShaderContentControl>

Notice that the implementation is a subclass of ContentControl. My original idea was to make a Blend Behavior, but to make this work I had to modify the visual tree. While this would be possible from a Behavior, it wasn’t optimal and would cause unexpected issues to the consuming developer. The ShaderContentControl will capture the UIElement set to the Content and use that as the target for the RenderTargetBitmap. Within the ShaderContentControl template is an <Image/>, that sits directly behind the set ShaderContentControl’s content. This is used to render the D3D content. Here is what the ShaderContentControl’s template looks like:

<ControlTemplate TargetType=”ContentControl” xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation&#8221; xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”&gt;

<Grid>

<ContentControl UseLayoutRounding=”False”

Width=”{Binding Content.ActualWidth, ElementName=contentShim}”

Height=”{Binding Content.ActualHeight, ElementName=contentShim}”

HorizontalAlignment=”{TemplateBinding HorizontalContentAlignment}”

VerticalAlignment=”{TemplateBinding VerticalContentAlignment}”>

<Image x:Name=”rendererImage”

Stretch=”Fill”

IsHitTestVisible=”False”/>

</ContentControl>

<ContentPresenter x:Name=”contentShim”

Opacity=”0.0″

ContentTemplate=”{TemplateBinding ContentTemplate}”

ContentTransitions=”{TemplateBinding ContentTransitions}”

Content=”{TemplateBinding Content}”

HorizontalAlignment=”{TemplateBinding HorizontalContentAlignment}”

VerticalAlignment=”{TemplateBinding VerticalContentAlignment}”/>

</Grid>

</ControlTemplate>

Keeping the <Image/> “perfectly” synchronized with where the UIElement would be rendered proved pretty difficult. With most shaders, like a Hue/Saturation modifier, one would only have to place the <Image/> behind the UIElement and bind to its size. The problem comes when you have shader effects that draw larger than the original sample. Shader’s such as drop shadow and blur are good examples of this. If this is not handled, the D2D surface is a natural clipping area and your blur clipped, which sucks. To overcome this, a shader can supply a padding value which supplies how much the shader will grow on the left, right, top and bottom and the ShaderContentControl will handle the rest.

Performance?

Performance is always a big concern but this is one of those limitations. Though the RenderTargetBitmap in XAML is the fastest relative to all other implementations, it still accounts for almost all the CPU overhead here. On top of that, RTB will map GPU memory to system memory to retrieve the pixel buffer, just to be sent back to the GPU by me. The other issue, there’s no way to tell a UIElement has visually changed, so a constant re-rasterization needs to be done (in some cases). That all being said, it does perform well, but there’s gotchas:

The ShaderContentControl has a property called “EffectRenderStrategy”, which can take 3 enums:

  1. RenderingEvent – Renders on every CompositionTarget::Rendering event
  2. Conservative – Renders when a layout updated has occurred or the pixel shader has been invalidated
  3. Manual – Renders only when ShaderContentControl->Render is called

Known Issues?

  1. RenderTargetBitmap::RenderAsync has issues with rendering certain elements.
  2. RenderTargetBitmap::RenderAsync has a problem with items in a ScrollViewer, such as a ListBox. I have a repro example here.
  3. Shaders are not extensible outside the library. This could be resolved, but I may have to do a lot of work abstracting Direct2D or making it more ABI friendly.
  4. This is a project-to-project library at the moment. I may make it an extension SDK later
  5. I only wrote a drop shadow and blur effect. Other effects coming soon.

Where do I get it?

https://xamlfx.codeplex.com/

3 comments

  1. Pingback: Windows App Developer Links – 2013-11-28 | Dan Rigby
  2. Joshua Heller

    Hi. Thanks for your library. I have a problem with the following set up. I want a dynamically loaded image in a FlipView shown full screen. Unfortunately, It does not work if I don’t set the initial width and height of the Image object. I tried several things i.e. get ShaderContentControl and call render everytime the FlipView selection changes or the image is loaded. I tried to set the width and height of rendererImage manually as well.

    Here’s my XAML: https://gist.github.com/torfeld6/9602564

    Any Ideas?

  3. john

    Thank you very much for the sample! now that i think about it i just got a really crazy idea but i’m not sure if is going to be possible. Do you think we could use the desktop thumbnail api(dwm) to display thumbnails of running desktop applications on Windows Runtime?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s