It’s been a while since I’ve made a blog post about something I’ve played around with just for the heck of it.  With Away3D 4.1 Alpha pushed out yesterday, I decided to spend the day revisiting an old friend: skin rendering. Remember the blog post of skin rendering when Away3D 4 was still codenamed Broomstick over a year and a half ago? (ugh, time flies!) A lot has changed in the engine, and many new features have been added. It made sense to therefore build a new but similar example showing some newer additions, and highlighting how things were put together. There’s two variations of the demo:

• With a familiar face: Using the same Lee Perry-Smith model as before, for direct comparison (and because it’s free and I’m a stingy wretch)
• With a unfamiliar face: Using a free (and insanely detailed) model from TurboSquid because I’m getting a bit sick of the old head (no offence intended to Mr. Perry-Smith) And again, it’s free and I’m a stingy wretch!

As usual, click+drag to move the camera.

There’s source right here. There’s a bunch of boiler plate in there, but the only thing of interest in this case should be the initMaterial method in Main.as. I’ll highlight the material setup here.

Multi-pass materials

The demo is using a multi-pass material for a few reasons. It allows the shadow mapping to more complex, and it prevents the non-casting lights from being masked by the shadow map, allowing for more dramatic lighting effects.

Soft Shadows

With a SoftShadowMapMethod assigned to the material, it’s possible to – as the name suggests – cast soft shadows on the skin. The class was updated in Away3D 4.1 along with some other shadow map methods, allowing more samples to be taken in the shadow map. This way, the results are much smoother which is especially important when tweaking the method’s range property. It sets the distance with which shadow map samples are taken, effectively creating softer shadows (a setting you can tweak in the demos). Of course, the more samples you take, the more demanding your shader will become.

Fresnel Specular Highlights

This is identical to the old demo. It uses FresnelSpecularMethod to achieve the fresnel effect, causing stronger highlights at glancing viewing angles. The fresnelPower can be tweaked to change the viewing angle fall-off: higher values increase the fresnel effect.

Subsurface Scattering Through Gradient Diffuse Lighting

By using GradientDiffuseMethod, you can acchieve a very crude approximation to subsurface scattering. It allows you to pass a gradient image to define the colour and strength of the diffuse reflection for each light/normal-angle. The left of the texture (x = 0) contains the reflection for surfaces pointing away from the light, the right is the reflection when completely facing the light. This allows darker lighting to be made a bit lighter. In this case, by introducing a little red in the mid-values (see “embeds/diffuseGradient.jpg”), we can simulate the addition of scattered light into the final shaded colour. While it doesn’t create true translucency effects similar to SubsurfaceScatteringDiffuseMethod, it does create an organic softness when viewing the surface head-on, an effect that is as subtle as it is effective.

Lighting Set-up

The light set-up is very traditional, a directional light coming from above and the front (by default) with a bright blue point light to the right and a red point light to the left. It’s worth noting however that I’ve set the directional light’s specular property to a much lower value to prevent it from creating strong highlights. Much like a softbox, I’d imagine.

Enjoy tweaking!

If you’ve been paying attention to the Away3D blog, you probably already know that the 4.1 alpha has been released today, which has been my main fixation since September (when I wasn’t getting radiation poisoning). As mentioned in the release post, one of the new features is “multipass shading”. If you’re not into 3D rendering programming, you may be asking yourself: “¿Qué?”, so I would like to go a bit more in-depth into the whats, whys, and hows.

Multi-pass shading

Multipass rendering is simply executing different render calls (“passes”) for a single geometry. Strictly speaking, Away3D 4.0 has supported multiple passes since its inception; SubSurfaceScatteringDiffuseMethod caused a depth map pass to be added, and OutlineMethod introduces a new pass to render the outline seperately from the normal geometry. However, all lighting – and methods contributing to the colour of the final output pixel such as fog, rim lighting, etc – happened in a single pass. With shaders being limited in number of instructions and amount of registers, so are the amount of lights that can affect a surface as well as the complexity of any other piece of code, shadow mapping being a common victim. This is where 4.1 introduces multiple passes (hence multi-pass shading) in the form of TextureMultiPassMaterial and ColorMultiPassMaterial. For all intents and purposes, they work identical to their single-pass counterparts (TextureMaterial and ColorMaterial, respectively), but they automatically split up into different passes depending on the amount of lights and the use of shadow mapping. The results are blended together to form a correctly shaded surface. This way, there’s no strict upper limit for lights. Of course, drawing the geometry multiple times comes with its own cost, and you should take care not to go gung-ho just because you can!

Cascade Shadow Mapping

Having shadows in a separate pass greatly increases what you can do to increase shadow quality. Getting shadows to look great is always a bit of a challenge, and one of the techniques that’s popular is Cascaded Shadow Maps (a great explanation here). If you’re too lazy to read the msdn article, here’s the gist of it. Due to perspective projection, fragments close to the viewer generally suffer shadow aliasing, because a texel in the shadow map covers much more screen space closer to the camera than it does farther away. In fact, for distant fragments, the shadow map resolution is often too large since several shadow map texels map to the same screen fragment. So there’s not enough resolution in the front, and possibly too much in the back! Cascaded Shadow Maps try to solve this by splitting up the view frustum and rendering separate shadow maps for each segment. The closer to the viewer, the smaller the frustum segment should be so as to increase the projected resolution of the shadow map.

CSM in Away3D 4.1

As you’ll see in demo code, it’s pretty easy to get this technique to work. Just be sure to use a multi-pass material and assign CascadeShadowMapper to the light and a CascadeShadowMapMethod to the material (this is identical to how NearDirectionalShadowMapper and NearFieldShadowMapMethod work).

Away3D splits up the view frustum automatically, but it’s often a good idea to play around with different split ratios yourself. The values with the best results depend heavily on the scene, how close you allow the camera to get to shaded surfaces, and so on. This is done by simply specifying the ratio in the view frustum for each cascade level to reach, “0″ being the frustum’s near plane, “1″ being the far plane. Generally, you’ll want the last cascade level to reach to the far plane, thus passing “1″.

For example, when using 4 cascade levels, you could specify the splits as follows:

A small detail to note: because texture memory is a precious resource, Away3D internally only uses a single texture for all (up to 4) shadow maps.

The demo and source

The demo that was built to demonstrate the multi-pass materials features the Sponza model built and made available by Crytek. You can find the original at their download site. It’s a bit of a behemoth, so give it some time to load It allows you to play around with some of the shadow map settings, which should be pretty straightforward. At least, they will be when I get around to writing an upcoming blog post about shadow map filter types. Source for the demo can be found on Github.

If you were at my presentation at Reasons to be Creative (what were you doing, missing out on the great weather outside!) you may recognize the set-up. It’s the same as I used for the deferred rendering demo; however, this is not deferred rendering: we merely re-purposed the demo

Get the Away3D 4.1 alpha version in the dev branch on Github.

As mentioned in the previous blog post, the dev branch of Away3D 4.0 now has light probes. Light probes are essentially a special sort of light source that contain global lighting info for the scene location they’re placed at, encoded as cube maps. For diffuse lighting, these cube maps are simply indexed by the surface normal of the shaded fragment. For specular lighting, the view reflection vector is used. As such they’re a fast basic way of faking global illumination. The fact that it’s using cube maps means that the lighting would only be physically correct at the precise point where the light was sampled, but the results can still be convincing regardless. Several light probes can be placed in a scene; the material’s light picker will assign weights to each to determine the importance in the lighting step based on the rendered object’s position relative to each.

You can use normal lights alongside the light probes, and in case of the default material system, you can specify which lighting component (diffuse and specular) is affected by which light types (material.diffuseLightSources and material.specularLightSources accepting LightSources.LIGHTS, LightSources.PROBES,  or LightSources.ALL). This can be useful if you want to use light probes for diffuse global lighting, but want specular lights from traditional light sources without those affecting the diffuse light.

Pre-process step

A downside of this approach over conventional light sources is that it requires a pre-process step to generate the cube maps. To start with, you need to generate (HDR) cube map renders for every point where you’ll want to put a light probe. From this, two process steps can be applied: a diffuse convolution and a specular convolution, depending on which type of lighting the probe needs to support. These can be generated in the eminent Paul Debevec’s HDRShop. I’ll warn you, however, that these convolutions can be very time-consuming – but the results are very accurate. If you’re willing to sacrifice some accuracy and get your hands dirty, you can write your own “convolutor”. Be warned however, that Flash doesn’t support HDR in BitmapData or textures, so if you’re using BitmapData and Stage3D, you’ll end up with under-lit results due to the low dynamic range of “bright” light. Having said that, I’ll quickly outline the principles behind it, just in case it interests anyone.

Lighting functions are spherical functions (the domain is the unit sphere, being mapped to scalar values), so you’ll need to integrate the diffuse or specular lighting function over the unit sphere of surface normals/view reflections. In fact, integrating over the hemisphere is usually enough since there’s no light coming from inside objects in our case. But as I was lazy and it was much easier to just implement it for the entire sphere I’ll use that approach For basic (Lambertian) diffuse reflections, this semi-formally boils down to:

Where N is the surface normal, “ω_i” is the incident light vector, and L(ω_i) is the light colour coming from the direction ω_i. The part “max(N.ω_i, 0) L(ω_i)”, is in fact just the same as the diffuse lighting calculation done for point and directional lights (where L is a constant function). In this case, L(ω_i) is simply a sample from the HDR cube map we rendered before. The integral just means that we’re taking the diffuse contributions for all directions (ie: all vectors ω_i in the unit sphere S) and accumulating them. Finally, to be able to sample the incoming light for each normal in an environment map, we would need to perform this calculation for every point in that map (remember, cube maps coordinates are simply 3D vectors, in this case representing the surface normal N).

In theory, the integral means we’d be summing up infinite amounts of infinitesimal samples. In practice, we’ll be using discrete steps to solve this. That’s okay, since we only have finite amount of samples to work with anyway; 6*n² pixels to be exact, n being the size of the HDR cube map. So, the most accurate way of doing things would be to add up all the contributions from the source map, and do that for each pixel in the destination map. If we’re rendering to a cube map of size m, that means it’s an order of 6*m² * 6*n² = 36m²n² (6 sides of m*m, each pixel sampling 6 sides of n*n). This can easily become a very expensive operation. Luckily, there are some numerical methods to solve this faster (at the cost of accuracy, or what did you expect). The most useful in this case is definitely Monte Carlo integration, which allows less samples from the source material per pixel. There are many articles written about the subject, and a great introduction is found in this article by Robin Green.

Enough to get you cracking, I think

Demo

See the Cornell Boxness of it all!

Use the arrow keys to move the head, space bar to toggle the texture (shows the lighting impact), and finally click and drag to rotate the head.

The head model and textures are still by Lee Perry-Smith (who by now must feel pretty awkward appearing in random tech demos). Source for the demo can be found in the texture-refactor branch of the examples repo (which will be merged to master together with the dev branch).

The environment maps were made as outlined before using a custom generation tool, and placed close to each corner of the box where they were generated.  I was using Stage3D (and thus a low dynamic range) to generate them, so I had to do some touch-ups in Photoshop to get them to look right. I’m not done investigating the options for further development of such map generation tools, but who knows they’ll make it public some day. In any case, HDRShop is more reliable! Diffuse lighting is set to use light probes only, while specular lighting uses the point light only.

The demo also shows how to use point light shadows, rim lighting and light mapping for added static shading.

Since I’ve been working on Away3D 4.0 (Broomstick) for the past few months, it was pretty hard to find the energy to keep this blog up to date, and the promised part 2 of Hacking the Lite kept getting delayed. And delayed. And then delayed some more. Having access to the GPU also caused some debate on whether this series would still be relevant enough. However, after looking at my initial goals for it, I decided it is. After all, the idea was to inspire people to start playing around, and to get people to think in terms of using shaders. Most of the concepts outlined here can be used in Molehill as well. And of course, it’s still some time before we can use the GPU in actual commercial projects So on with the show! This part of the series will build heavily upon the previous part, so it’s perhaps a good idea to review that again.

Shadow maps

Casting shadows is often done using shadow maps. In this case, we’d render a “depth map” of the scene from the light’s point of view. This way, we have a texture containing the distances to the objects closest to the light for every projection ray (which is a single pixel in the shadow map). When rendering the scene, we project every point we are rendering to that shadow map. We can then compare the depth value of the point-to-be-rendered with the value in the shadow map. If it is further away from the light, it must be in shadow and should not be lit. This approach works well, but for current versions of Flash, it’s not very feasible. But, since we’re hacking, there’s other options!

Augmenting projective texture mapping

If you take a look at the schematic on the left (click to enlarge), showing where a shadow should fall, you can see it’s much like projection mapping. The sphere is being perspectively projected unto the floor plane with the light as the projection’s origin. This is the exact same projection that projects the light texture onto the objects. As you can see, if we project the sphere onto the light’s texture first, and project that in turn to the floor, the shadow region remains the same. As a result, we could actually simply project and draw the sphere into the light map before it is applied to the floor.

Order and form

To render a scene with multiple objects, we have to take care to render things in correct order. We need to draw the objects closest to the light first, because they will obviously be casting shadows on object further from the light. Starting out with a clean light map (ie.containing no shadows), we project it on the closest object and then draw that object’s shadow into the light map. Move on to the second closest object, light it, and draw the shadow again. And so on until the last object was reached.

That works great for convex objects, as long as they don’t intersect. For concave or intersecting convex objects, it’s possible that object A casts shadows on object B as well as the other way around. Since we’re rendering everything in a strict order, that’s simply not a possibility. However, this error should generally be acceptable. Another problem is self-shadowing for concave objects, which is impossible using this “cheat”.

Demo and source

Fake shadow mapping demo: really just an adapted version of the previous one.

The source is again on Google Code!