I updated my DirectCompute Mandelbrot fractal demo to be able to render using both CUDA and DirectCompute. The program is a simple Mandelbrot renderer, but you can dynamically switch between a CUDA and DirectCompute render.

Relevant Links

http://developer.download.nvidia.com/compute/cuda/sdk/website/Graphics_Interop.html#simpleD3D11Texture
The NVIDIA CUDA 3.2 SDK includes a DirectX11 interop sample “Simple D3D11 Texture” which I used as reference. For the 3.2 SDK, I think this is the only CUDA sample using D3D11. The SDK sample uses a ID3D11Texture3D as a CUDA resource while my Mandelbrot fractal program uses a ID3D11Buffer as a CUDA resource.

http://developer.nvidia.com/object/cuda-by-example.html
I mainly used the CUDA 3.2 SDK programming manual to learn CUDA, but I also read the NVIDIA “Cuda by Example” book on Safari Informit. The book is very easy to follow and I read it in one sitting. This book teaches the basics of CUDA in a simple language and I thought it was a good first book.

http://www.yakiimo3d.com/2010/02/02/directcompute-mandelbrot-fractal-viewer/
My old DirectCompute Mandelbrot fractal viewer. The program has a bug (oh no!) when the screen size dimensions are not divisible by the thread group size dimensions, which I fixed in my new CUDA interop demo.

Demo Notes

Took some timings to compare the CUDA DirectX11 interop and DirectCompute performances. Timings were taken on my Geforce GTX 460, Driver 266.58, Vista 64-bit SP2, CUDA SDK 3.2. Neither the CUDA nor DirectCompute implementations have been optimized. With better documentation, better tools and finer control over the program, I think the CUDA program has a better chance of good optimization.

The higher the iteration count, the more work that the compute shader has to do. When the iteration count is low, DirectCompute is faster. When the iteration count becomes higher, CUDA becomes faster than DirectCompute. I’m assuming this means that with my current code, the CUDA kernel execution is faster, but that there is a fixed cost for DirectX11 interop that makes CUDA initially slower.

http://nvidia.fullviewmedia.com/gdc2011/agenda.html
On Friday, I watched the NVIDIA GDC2011 “GPU Radiosity: Porting the Enlighten runtime to CUDA” presentation, and around 28:23, the speaker mentions that “Switching between D3D and CUDA is expensive (it’s a power cycle!)”. I’m guessing this power mode switch cost is what makes CUDA initially slower in my Mandelbrot fractal program. I watched the entire Enlighten presentation and it was very interesting. The stream had some technical info, but if you are curious about Enlighten tech, there is a DICE&Geomerics Siggraph 2010 presentation that contains even more detailed technical information about Enlighten http://advances.realtimerendering.com/s2010/index.html.

Demo

Source Code & Binary
http://yakiimo3d.codeplex.com/releases/view/62087

I updated my simple Temporal AA demo to include an implementation of Temporal AA without blending. A lot of times, in technical 3D forums, you read about Temporal AA not showing up in screenshots. If you blend jittered frames like in my previous demo then Temporal AA is going to show up in screenshots. However, it’s also possible to implement Temporal AA without manual blending, and in that case, the AA is not going to appear in screenshots. Over the weekend, I implemented this other version of simple Temporal AA.

CodePlex link for my program’s source code and binary are provided at the end of the article.

Relevant Links

1) http://en.wikipedia.org/wiki/Radeon_R420
The Radeon X700-X850 series apparently featured Temporal AA without blending. Don’t think it’s supported on newer graphics cards as my HD5750 with Catalyst 10.10 doesn’t show an option.

2) http://techreport.com/articles.x/6672/22
This Tech Report article has a nice explanation of the Radeon Temporal AA implementation. Mulitisampling patterns are alternated each frame. If vsync is enabled and the app can maintain 60fps, even without manual blending, the eyes will interpolate the jittered frames giving an illusion of higher-sampled anti-aliasing. In the Radeon implementation, 2x MSAA using temporal AA will give an illusion of 4x AA.

3) http://forum.beyond3d.com/showthread.php?t=46241
Beyond3D’s “List of Rendering Resolutions + basics on hardware scaling, MSAA, framebuffers” thread mentions both Temporal AA with and without blending.

4) http://www.yakiimo3d.com/2010/09/28/dx11-perspective-matrix-jittering-temporal-aa/
My implementation of Temporal AA with blending. A simple implementation without reprojection, so you get ghosting/blurring.

Implementation Details

The implementation is the same as my last Temporal AA implementation except that I don’t manually blend the previous and current frame, but let the eye do the interpolation. I added a combobox for selecting the vsync interval. Only vsynced 60fps gives good results; non vsynced 60+fps, vsynced 30fps, vsynced 20fps all give pretty terrible edge flickering. With blending, the artifact is blurring/ghosting, without blending, the artifact seems to be edge flickering.

Unlike the Radeon implementation, I’m not alternating MSAA patterns, but just alternating jittered scene renders, so the quality should be equivalent to 2x SSAA (like my previous blended Temporal AA implementation). In the demo, the Temporal AA without blending looks a little bit worse than Temporal AA with blending because I decreased the sub-pixel offset to 0.15 from 0.25. Even at a vsynced 60 fps setting, I can sometimes still see an occasional edge flicker.

Demo

Source Code & Binary
http://yakiimo3d.codeplex.com/releases/view/56973

I would recommend viewing the demo in full-screen mode as this seems to give a more stable frame rate, and less edge flickering. I also fixed my texture mipmap settings and set the anisotropic filter to 16x. Even at this setting, you can still see a tiny bit of reduction in texture aliasing (without noticeable texture blurring from the SSAA, I think) in some parts of the scene, but it’s not significant and I guess that’s not the main point of Temporal AA.

Sorry for the download size. My mesh model data, borrowed from the DirectX SDK samples, is pretty big. Like my previous Temporal AA implementation, DX11 isn’t necessary for this technique, but my hobby coding is now done using DX11, so that’s what the demo uses.

Final Words

While I was working on my last Temporal AA demo, another Japanese programmer told me that he saw very little texture aliasing reduction in his Temporal AA test, but I foolishly didn’t check my code for bugs. Hopefully, I didn’t make any stupid mistakes this time!

Temporal anti-aliasing algorithms are algorithms where you use samples from previously rendered frames in order to perform a temporal form of supersampling. There are different ways to implement temporal AA, and over the weekend, I wrote an implementation of the simple perspective matrix jittering temporal AA. This type of temporal AA is mentioned in the Beyond3D forums here (http://forum.beyond3d.com/showthread.php?t=46241).

CodePlex link for my program’s source code and binary are provided at the end of the article.

Halo: Reach Temporal AA

http://www.eurogamer.net/articles/digitalfoundry-halo-reach-tech-analysis-article (AA is covered on Page 3.)
Digital Foundry recently published a tech analysis article on Halo: Reach. Based on frame capture analysis and the behavior of the anti-aliasing, they speculated that the anti-aliasing used in Halo: Reach is some kind of temporal anti-aliasing.

CryEngine3 Temporal AA

http://advances.realtimerendering.com/s2010/index.html
At Siggraph 2010′s “Advances in Real-Time Rendering in 3D Graphics and Games” course, Crytek presented a talk “CryENGINE 3: Reaching the Speed of Light” in which they gave some details about their anti-aliasing implementation. Their anti-aliasing is a hybrid solution with 2 different AA algorithms applied to near and distant objects. For near objects, an edge-based post process AA is applied. For distant objects, temporal AA (the temporal reprojection with cache miss approach) is applied.

The below papers were given as references for temporal reprojection in Crytek’s talk.

Accelerating Real-Time Shading with Reverse Reprojection Caching (ACM SIGGRAPH Symposium on Graphics Hardware 2007)
http://www.cse.ust.hk/~psander/
http://www.cse.ust.hk/~psander/docs/reproj2.pdf (pdf)
and
Spatio-Temporal Upsampling on the GPU (I3D 2010)
http://www.mpi-inf.mpg.de/~rherzog/
http://www.mpi-inf.mpg.de/~rherzog/Papers/spatioTemporalUpsampling_preprintI3D2010.pdf (pdf)

Screen Captures

No AA

MSAA 2x

MSAA 4x

Temporal AA 2x

Comments On the Screen Captures

(2010/10/02 added comment)
http://www.yakiimo3d.com/2010/10/02/temporal-aa-texture-mipmap-settings/
Texture filtering was not properly set up when I took the above pics. If texture filtering is properly enabled, the aliasing inside the textures becomes gone. With mipmapping-enabled, the effects of temporal AA on texture aliasing becomes minimal.

My implementation of temporal AA blends together 2 samples and should be roughly equivalent to a 2 sample SSAA in terms of quality. For comparison, I also implemented hardware MSAA 2x and hardware MSAA 4x. In the above screenshots, you can see that for long edges, MSAA 2x and temporal AA 2x are around the same quality. MSAA 4x results in much cleaner long edges than both temporal AA and MSAA 2x. If you look at the inside of the textures, you will notice that temporal AA 2x results in much cleaner straight lines and less overall aliasing compared to both MSAA 2x and MSAA 4x. Temporal AA is a form of supersampling and is able to reduce pixel shader aliasing on surfaces and not just jaggies on edges. However, for the above picture, if texture mipmapping had been properly enabled, the hardware texture filter would have taken care of the aliasing.

In my Japanese blog, I posted screenshots comparing the different AA algorithm’s results on a grid-patterned texture. Temporal AA works really well for this type of high frequency texture. While neither MSAA 2x nor MSAA 4x are able to reduce the aliasing much, temporal AA successfully reduces the moire pattern and the aliasing.
http://d.hatena.ne.jp/yakiimo02/20100926/1285523408

MSAA uses multiple depth samples, but only a single pixel shader sample, while temporal AA, like SSAA, uses multiple pixel shader samples. This means that temporal AA effectively operates at a high resolution compared to screen size and allows the algorithm to reduce aliasing on detailed polygon surfaces requiring more info than a pixel can hold.

The perspective matrix jittering temporal AA algorithm is cheap and works great on still pictures. However, it has problems once the camera starts moving (especially in sideways motions) because the difference between the previous frame’s pixels and the current frame’s pixels becomes large and as a result when the 2 frames are blended, the resulting screen is blurry with slight halo traces of the last frame.

Implementation Details

http://glprogramming.com/red/chapter10.html
Chapter 10 of the OpenGL Redbook explains the OpenGL accumulation buffer AA algorithm.
http://www.cse.msu.edu/~cse872/tutorial5.html
Found out that the Redbook explains the OpenGL accumulation buffer AA algorithm from this Michigan State University’s CSE 872 Advanced Computer Graphics class’s tutorial.

Chapter 10 of the OpenGL Redbook has an explanation on using perspective matrix jittering in order to perform anti-aliasing through the accumulation buffer. In the Redbook example, they re-render the scene n-sample times, each time with the perspective matrix jittered by a small sub-pixel amount, and blend the result into the OpenGL accumulation buffer. This subpixel n-sample blending is equivalent to SSAA and gets you an anti-aliased image. The technique requires re-drawing the scene n-sample times within a single frame and is prohibitively costly for most real-time applications.

My temporal AA implementation is a 2-sample perspective matrix jittered AA with one sample taken from the previously rendered frame and one sample from the current frame. The implementation requires an extra rendertarget the same size as the framebuffer in order to store the previous frame’s color buffer (assuming the current frame’s rendertarget is already necessary and so free). Even frames and odd frames are renderered with a different subpixel jitter value and are blended together in a fullscreen post process pass. Same as the OpenGL accumulation buffer AA, the temporal blending of 2 subpixel jittered color buffers should give similiar results to SSAA 2x. The only tricky part about the implementation for me was figuring out how to create a perspective projection matrix that jitters the rendered image by a screenspace subpixel amount. The function that performs this calculation is shown below.

Perspective Matrix Jittering Temporal AA Demo

The background model used in the demo is the “powerplant” sdkmesh model included with the DirectX June 2010 SDK. It’s used in the DX11 Cascaded Shadowmaps demo.

In my demo, the frame sometimes jumps when you are moving the camera. I use the DXUT CFirstPersonCamera and same thing occurs for the Cascaded Shadowmaps demo and is not an artifact of temporal AA.

Source Code & Binary
http://yakiimo3d.codeplex.com/releases/view/53001

Conclusions

The perspective matrix jittering temporal AA is easy to implement, cheap and provides anti-aliasing qualities similiar to SSAA 2x. However, the current implementation suffers from distracting screen blurring once the camera starts moving. I haven’t really read up on temporal reprojection yet, but it might be interesting to implement AA using it and see how the results compare to this implementation.

I extended my DX11 OIT demo (http://www.yakiimo3d.com/2010/07/19/dx11-order-independent-transparency/) to support MSAA.

CodePlex link for my program’s source code and binary are provided at the end of the article.

Relevant Links

1) http://www.yakiimo3d.com/2010/07/19/dx11-order-independent-transparency/
Blog article for my original DX11 OIT demo. My original demo has no support for MSAA, so jaggies are very noticeable along the edges of the colored quads.

2) http://developer.amd.com/documentation/presentations/Pages/default.aspx
ATI’s conference presentation list contains a link to their “GDC 2010: OIT and GI using DX11 linked lists” presentation ppt. My original demo and this MSAA supported version are both based on this presentation.

Screen Captures

From top to bottom. MSAA none. MSAA 2x. MSAA 4x. MSAA 8x. I notice big quality improvements between none, 2x and 4x, but not too much between 4x and 8x. I think the anti-aliasing starts to look pretty nice around 4x. Notice that with increased sampling, the image quality improves, but the fps also drops.

Implementation Details

Not too many changes were necessary to the original demo in order to add MSAA support. The main change was adding the SV_COVERAGE and SV_SAMPLEINDEX semantic parameters to my pixel shader input structures and using their information.

Changes to StoreFragments.hlsl

I added a nCoverage SV_COVERAGE parameter to give me information about which samples are covered by the current fragment. For example, bit 1, 4 of nCoverage are set if samples 1 and 4 are covered. Now, in addition to storing each fragment’s color and depth, I also store the fragment’s sample coverage information.

Changes to SortFragmentsAndRender.hlsl

The SV_SAMPLEINDEX semantic was added to DX10.1 and can be used in DX11. By adding a pixel shader input parameter with the SV_SAMPLEINDEX semantic, it’s possible to make the pixel shader run once per sample instead of once per pixel. By running the sorting & blending pixel shader per sample and discarding the samples not covered by the fragment, it’s possible to get MSAA anti-aliasing even with the StructuredBuffer input fragment data.

OIT with MSAA Demo

Source Code & Binary
http://yakiimo3d.codeplex.com/releases/view/49570
The demo starts with a default setting of 4x MSAA. Open the Change Device menu and change the MSAA setting using the Multisample Count combobox.

Comments

This demo was my first time using the SV_SAMPLEINDEX pixel shader semantic and processing pixel per sample instead of per pixel. It’s apparently useful for other stuff such as deferred rendering MSAA, so I’ll probably find myself using it again some time in the future. Also, I bought the XBOX Live Arcade game Limbo and played it while taking a break from writing this blog post. From what I’ve played so far, it’s a dark, but very artistic and beautiful game. So far, I’m glad I bought it.

This is my implementation of the ATI DX11 linked list OIT algorithm. The DirectX11 SDK OIT sample is pretty slow (a compute shader usage sample), but have read from numerous sources that the ATI OIT algorithm is pretty fast (it is), so I decided to try implementing it. Lots of people have already implemented ATI’s linked list OIT and have made public their implementations so I had a lot of good references for my simple implementation.

CodePlex link for my program’s source code and binary are provided at the end of the article.

Relevant Links

1) http://developer.amd.com/documentation/presentations/Pages/default.aspx
ATI’s conference presentation list contains a link to their “GDC 2010: OIT and GI using DX11 linked lists” presentation ppt. This demo is an implementation of the algorithm described in this presentation.

2) http://www4.atword.jp/cathy39/category/direct3d11/oit-direct3d11/
GPU Pro contributor Kaori Kubota-san’s linked list OIT implementation. She walks through the linked list OIT implementation pretty carefully and her explanations are interspersed with source code. Couldn’t find sample code for stuff like D3D11_BUFFER_UAV_FLAG_COUNTER, so her explanations were very helpful. My implementation is based on the ATI presentation ppt and her blog posts.

3) http://orenk2k.blogspot.com/2010/03/oit-order-independent-transparency.html
A blog post about a linked-list OIT implementation. Good explanations.

4) http://sites.google.com/site/monshonosuana/directxno-hanashi-1/directx-110
A Japanese graphics programming website recently posted a linked list OIT implementation. Good explanations (in Japanese) with source code and binary.

5) http://www.uraldev.ru/articles/id/36
Another linked list OIT implementation. Article is in Russian, so I used google translate to read it. Nice article with source code and binary provided. Learned about the implementation from the author’s blog post comment here http://www.wolfgang-engel.info/blogs/?p=116

6) http://joescg.blogspot.com/2010/01/oita-buffer-demo.html
Another Russian OIT implementation. Good explanations read using google translate again. Learned about the implementation from here http://www.geeks3d.com/forums/index.php/topic,861.0.html

7) http://www.yakiimo3d.com/2010/06/19/dx11-high-quality-global-illumination-rendering-using-rasterization/
The DX11 linked list technique can be used to speed up algorithms besides alpha blending. Toshiya Hachisuka-san’s “High-Quality Global Illumination Rendering Using Rasterization” algorithm can be sped up using ATI’s StructuredBuffer linked list technique.

A Screen Captures From My OIT Demo

Regular alpha blending using ID3D11BlendState. Notice that the far section of the blue quad is incorrectly rendered over the red quad. Since each colored quad is drawn in a single draw call, it’s not possible to sort the geometry to get correct alpha blending.



Rendered with the OIT algorithm. In the first pass, all pixel fragments are stored into a large StructuredBuffer. In the 2nd pass, the stored pixel fragments are read and sorted per-pixel inside a pixel shader program and everything is correctly rendered.

Implementation Details

Other websites already explain the linked list OIT algorithm, so not going to do that in this blog post. On my ATI HD5750, with the DirectX SDK OIT sample, I get a fps of around 10 fps at a screen size of 320×240. With my ATI OIT implementation, I get a fps of around 700 fps at a screen size of 640×480. With regular ID3D11BlendState alpha blending, I get a fps around 3000 fps at a screen size of 640×480.

OIT Demo

Source Code & Binary
http://yakiimo3d.codeplex.com/releases/view/49213

Comments

It’s been over 3 months since my last demo. For the past month, I had been feeling somewhat down, but now I feel much better, so maybe I will be more productive. Not sure what I want to implement next. Maybe a translucent material demo based on this OIT demo. Or maybe experiment with DX11 tesselation and implement a DX11 Phong Tesselation demo.