As mentioned in the previous blog post, the dev branch of Away3D 4.0 now has light probes. Light probes are essentially a special sort of light source that contain global lighting info for the scene location they’re placed at, encoded as cube maps. For diffuse lighting, these cube maps are simply indexed by the surface normal of the shaded fragment. For specular lighting, the view reflection vector is used. As such they’re a fast basic way of faking global illumination. The fact that it’s using cube maps means that the lighting would only be physically correct at the precise point where the light was sampled, but the results can still be convincing regardless. Several light probes can be placed in a scene; the material’s light picker will assign weights to each to determine the importance in the lighting step based on the rendered object’s position relative to each.

You can use normal lights alongside the light probes, and in case of the default material system, you can specify which lighting component (diffuse and specular) is affected by which light types (material.diffuseLightSources and material.specularLightSources accepting LightSources.LIGHTS, LightSources.PROBES,  or LightSources.ALL). This can be useful if you want to use light probes for diffuse global lighting, but want specular lights from traditional light sources without those affecting the diffuse light.

Pre-process step

A downside of this approach over conventional light sources is that it requires a pre-process step to generate the cube maps. To start with, you need to generate (HDR) cube map renders for every point where you’ll want to put a light probe. From this, two process steps can be applied: a diffuse convolution and a specular convolution, depending on which type of lighting the probe needs to support. These can be generated in the eminent Paul Debevec’s HDRShop. I’ll warn you, however, that these convolutions can be very time-consuming – but the results are very accurate. If you’re willing to sacrifice some accuracy and get your hands dirty, you can write your own “convolutor”. Be warned however, that Flash doesn’t support HDR in BitmapData or textures, so if you’re using BitmapData and Stage3D, you’ll end up with under-lit results due to the low dynamic range of “bright” light. Having said that, I’ll quickly outline the principles behind it, just in case it interests anyone.

Lighting functions are spherical functions (the domain is the unit sphere, being mapped to scalar values), so you’ll need to integrate the diffuse or specular lighting function over the unit sphere of surface normals/view reflections. In fact, integrating over the hemisphere is usually enough since there’s no light coming from inside objects in our case. But as I was lazy and it was much easier to just implement it for the entire sphere I’ll use that approach For basic (Lambertian) diffuse reflections, this semi-formally boils down to:

Where N is the surface normal, “ω_i” is the incident light vector, and L(ω_i) is the light colour coming from the direction ω_i. The part “max(N.ω_i, 0) L(ω_i)”, is in fact just the same as the diffuse lighting calculation done for point and directional lights (where L is a constant function). In this case, L(ω_i) is simply a sample from the HDR cube map we rendered before. The integral just means that we’re taking the diffuse contributions for all directions (ie: all vectors ω_i in the unit sphere S) and accumulating them. Finally, to be able to sample the incoming light for each normal in an environment map, we would need to perform this calculation for every point in that map (remember, cube maps coordinates are simply 3D vectors, in this case representing the surface normal N).

In theory, the integral means we’d be summing up infinite amounts of infinitesimal samples. In practice, we’ll be using discrete steps to solve this. That’s okay, since we only have finite amount of samples to work with anyway; 6*n² pixels to be exact, n being the size of the HDR cube map. So, the most accurate way of doing things would be to add up all the contributions from the source map, and do that for each pixel in the destination map. If we’re rendering to a cube map of size m, that means it’s an order of 6*m² * 6*n² = 36m²n² (6 sides of m*m, each pixel sampling 6 sides of n*n). This can easily become a very expensive operation. Luckily, there are some numerical methods to solve this faster (at the cost of accuracy, or what did you expect). The most useful in this case is definitely Monte Carlo integration, which allows less samples from the source material per pixel. There are many articles written about the subject, and a great introduction is found in this article by Robin Green.

Enough to get you cracking, I think

Demo

See the Cornell Boxness of it all!

Use the arrow keys to move the head, space bar to toggle the texture (shows the lighting impact), and finally click and drag to rotate the head.

The head model and textures are still by Lee Perry-Smith (who by now must feel pretty awkward appearing in random tech demos). Source for the demo can be found in the texture-refactor branch of the examples repo (which will be merged to master together with the dev branch).

The environment maps were made as outlined before using a custom generation tool, and placed close to each corner of the box where they were generated.  I was using Stage3D (and thus a low dynamic range) to generate them, so I had to do some touch-ups in Photoshop to get them to look right. I’m not done investigating the options for further development of such map generation tools, but who knows they’ll make it public some day. In any case, HDRShop is more reliable! Diffuse lighting is set to use light probes only, while specular lighting uses the point light only.

The demo also shows how to use point light shadows, rim lighting and light mapping for added static shading.

Molehill, Adobe’s new API to leverage the GPU for rendering, was made public some time ago in the form of an “incubator” Flash Player release, and along with it we released an early pre-alpha version of Away3D 4.0, codenamed “Broomstick”. I’ve been working on the new engine for quite some time now, and since the release I hadn’t had the time to post any more information aside from what was shown on the blog post. Time to change that, I’d say!
Before you do anything else, you’ll need to install the Incubator Flash Player, because the demos will be using that. Secondly, there’s no guarantee that anything related to Away3D 4.0 will be exactly the same in the final release, as it is of course an early alpha. Finally, source for all demos is in the usual Away3D repository. They’re a bit messy, since they were actually just quick test files to see if everything still works

Subsurface scattering

Subsurface scattering is basically what happens to light when it travels through a translucent surface. Instead of either simply reflecting off a surface or passing through it, the light will enter the material, reflect a good couple of times before exiting the surface again and reaching the eye. This behaviour causes a general blurring of the diffuse lighting on an object and allows light to shimmer through less “thick” regions. Just think about how a candle’s flame shines through the wax.

There’s a few rendering techniques that perform subsurface scattering to some extent. A popular approach is to render the diffuse lighting to a texture and simply blur it slightly. However, I find that this approach doesn’t allow light to travel through an object and so I opted for another approach: using depth maps. Similar to shadow mapping, a depth map is rendered from the point of view of the light, and when rendering, we compare depths of the rendered pixel to the value in the depth map. Instead of using this to determine whether a pixel is in shadow or not, we can use the difference between these values to determine the “thickness” of the object, or how far light has to travel through it to be able to reach the eye (see my amateurish schematic on the right ).

Depending on this distance, we can colour the pixel differently. We define a colour the light gets by traveling through the object (this depends on the actual material), and the final pixel’s colour ranges from this “scatter colour” to the material’s (texture) colour, depending on how thick it is.

This technique can be used to make materials look like wax, such as in this demo.

Advanced Skin Rendering

Skin is in fact a translucent object, so we can use subsurface scattering to increase the realism, as usual rendering techniques often create a harsh or clay-like effect. Instead, subsurface scattering softens the general shading and allows light to travel through more translucent regions such as the ears, softer parts of the nose, etc.
To further increase the realism, the specular highlights are modulated using a fresnel term. As with most objects in reality, the reflectivity of an object changes depending on the view angle, and with it the strength of the highlights. When observing skin head-on, there’s usually not a strong direct light reflection, but it can usually be perceived on shallow angles.
The result of both can be seen here.

In Away3D 4.0

First of all, to get started using Broomstick, check out this article by John Lindquist. He did a great job explaining how to get started!
Default materials in Away3D 4.0 are currently implemented using “methods”, which are properties of a material that define how the shaders work in a modular fashion. This way, there’s no need for a billion variation of Material classes for each combination. For subsurface scattering, we need to change the diffuseMethod property of a material and assign an instance of SubsurfaceScatteringDiffuseMethod. For the fresnel specular highlights, we assign an instance of FresnelSpecularMethod to the specularMethod property. It’s all in the source, so knowing this, you should be able to make sense of what’s going on in there

Demos

In case you skipped through the rest of the article, here’s a demo recap. Loading times may be long, as the models are pretty big

Wax Statue : Illustrating subsurface scattering to generate a material resembling wax.
Head rendering: Using subsurface scattering and a specular fresnel term to increase realism in skin rendering. Press any key to toggle the subsurface scattering, so you can see the difference. Note the light shining through the ears, for example.

Finally, something new! But no, it’s not any of the bigger things I’ve been hinting at before. However, I’ve been hard at work at those (and other) projects for months, and ended up with tunnel vision and an annoying rut. I really needed to do something smaller again for some immediate visual fun. When Bartek showed me something on Twitter, I decided to take a step back from all the work, dig up some half-year old code and finish it asap. And so I did: that is a Fresnel Pixel Bender shader for Away3D, presented in the FresnelPBMaterial!

I’ve explained it before, but I’ll give a more general explanation this time around. Fresnel shading mimics the way light reflects off of (often translucent) surfaces such as water or glass. Looking head-on, you can see its proper surface (or see through it with refraction). However, if seen from a shallow angle, it reflects light (just imagine looking over a large body of water, or the mirage effect on the road on hot summer days).

Demos

I’ve thrown together some rather random demos to illustrate the point. There’s source for both. (A word of warning: apparently, on some FP10.1 betas, Pixel Bender causes it to throw an invalid input exception, which seems to be bogus.)

• Glass and marble head: Yes, the head appears to be floating in the middle of a cathedral of some blurry sort. Click to cycle through different property settings: glass, alien marble (I’m sure that’s a thing), and plain marble.
• Water surface: Using an animated normal map to illustrate how water could be rendered while using the fake refraction. It also shows the WaterMap util.

Another word of warning: slight implementation details might change in the next few days/weeks, but it should be nothing too dramatic!

I’ll probably be doing something more with this when I have the time, but nothing that would likely make it to the repository

It’s no secret that I like graphics. It’s the main reason why I play video games and it’s the main reason I got into programming. So obviously I was delighted to be invited and to join to the Away3D team last month. Inspired by my earlier Stok3d project (now on the backburner for a bit), I set off to create similar normal mapped pixel shaders, this time in full 3D. After some rough first patches (Stok3d was pretty simple since it only used DisplayObjects, flat planes), things have luckily shaped up, leading up to the first release!

The current state is a dot-release (3.4.2), so the exact implementation might still change while we’re working towards a shiny new 3.5.

The new shaders

So what is the difference with the previous shaders? Using Pixel Bender, the shading is calculated for every pixel in the texture, resulting in much more detailed and realistic lighting or reflections. Each shader requires an object-space normal map, which you can use to add detailed shading information without increasing the polycount of the mesh.

The shaders come in three flavors: environment map shaders, and single- and multi-pass shaders. Single-pass shaders take one PointLight3D and any AmbientLight3D on the scene to calculate the lighting, whereas multi-pass takes any number of lights of any type (AmbientLight3D, DirectionalLight3D, and AmbientLight3D). Important to note, tho, is that every light adds another pass and will be slower. Of course, if you’re only using 1 light, always use single pass.

Check out the following classes in Google Code:

• SphericEnvMapPBMaterial:  spheric environment mapping
• Cubic EnvMapPBMaterial: cubic environment mapping

• DiffusePBMaterial: Single-pass, adds diffuse lighting to the texture
• PhongPBMaterial: Single-pass, adds diffuse lighting and specular highlights, with support for specular maps
• SpecularPBMaterial: Single-pass, adds only highlights – can be used in combination with Prefab3D‘s prebaked lighting to create view-dependent specular highlights

• DiffuseMultiPassMaterial: Diffuse shading with multiple light sources
• PhongMultiPassMaterial: Phong shading with multiple light sources

Demos

Enough explanations, time for some demos! Right-click to view source:

• Cubic environment mapping
• Single pass shading: showing how a detailed normal map and specular map can enrich the shading
• Multi pass shading: using 3 lights (directional moonlight, firefly, and a flickering flame) and ambient

That’s it, enjoy! Feel free to drop by the mailing list for questions or read the official announcement! For now, I need to get some sleep before Flash on the Beach kicks off

Last week I posted an example of Environment Mapping using FP10′s native 3D and Pixel Bender. The reactions were quite positive, which motivated me to push the concept a bit further and create more shaders using Pixel Bender. These new additions all work in the same fashion, ie. as Filters which need to be updated whenever the target object (or if provided: light position) changes.

I created a project on Google Code for this, dubbed Stok3d (as I was positively stoked at that specific point in time). It’s a mini-library at this point, but in the future there’s potentially more to it than shaders; Z-sorting for instance: although it has been done already, it wouldn’t be a bad idea to create something specifically for Stok3d and have more functionality in one place. But… zat’s for ze futuah! At least for now, it will be easier to commit bugfixes and updates.

Demos

In the order from low cpu usage (and less visually interesting) to high cpu usage (and more interesting):

• Diffuse Shading: Basic shadowing, no specular highlights or normal map support
• *Phong Shading: Phong shading with support for normal and specular maps
• *Environment Mapping with Phong Shading: Combination of cubic environment mapping and Phong Shading, supporting normal and specular maps.

* Although Stok3d is distributed under the GNU GPLv3 license, the textures are NOT covered by this license. In particular, the blast door and hangar door textures, normal maps, and specular maps are made by Florian Zender (http://www.florianzender.com) and are used with his kind permission. Check out his work, it’s quite impressive!

Source

The source for the examples as well as the library can be found on Google Code. It’s available over svn or as a downloadable archive.

Since Flash Player 10 introduced native 3D functionality, the world of “postcard 3D” rejoiced as doing simple 3D tasks became much easier… Just as long as it did not involve z-sorting or shading effects.

The z-sorting has been tackled before, and here’s an attempt at implementing the second idea. Partly out of boredom and partly because I needed it as a first step towards another experiment, I created a simple cubic environment map effect, just in case you’d want your surfaces to be reflective. The filter itself is using Pixel Bender – what a surprise!

There’s some other possibilities I’m thinking of (phong shading for example), but it’d be good to know if there actually is any interest in stuff like this. I don’t want to be wasting my time either

Cubic Environment Maps

Environment mapping is essentially a fake but relatively fast way to give the illusion of surface reflection. There are two common approaches: spherical and cubic mapping. The first maps the environment texture surrounding the scene on a sphere before mapping it onto the surface, the latter does it using a cube. Spherical mapping is usually faster and easier to use (it only requires 1 or 2 textures instead of one for every face of the cube), but sadly it doesn’t look very natural on a flat surface. Cubic mapping, on the other hand, looks better and is more commonly used these days. For more information, check this article on wikipedia.

So what do you need? Basically, you need to find a cube map and divide it up into 6 seperate images as illustrated below. You can assign these images to the EnvMapFilter class constructor.

When changing the surface position, scale or rotation, be sure to call the update method and reassign the filter to the target DisplayObject. All this is shown in the source.

Normal maps

It’s not a requirement, but if you want, you can assign a normal map to the filter. For every pixel, normal maps indicate the slope of the surface at any point and cause the reflections to be distorted. This gives the surface the appearance of having relief. A quick search on google or some texture libraries should yield plenty of them

Blabla yeah whatever!

Okay okay, enough chatter.

• Demo – just follow the instructions (FP10.0.22 required, so you might get an upgrade request)
• Source

Enjoy!