As mentioned in the previous blog post, the dev branch of Away3D 4.0 now has light probes. Light probes are essentially a special sort of light source that contain global lighting info for the scene location they’re placed at, encoded as cube maps. For diffuse lighting, these cube maps are simply indexed by the surface normal of the shaded fragment. For specular lighting, the view reflection vector is used. As such they’re a fast basic way of faking global illumination. The fact that it’s using cube maps means that the lighting would only be physically correct at the precise point where the light was sampled, but the results can still be convincing regardless. Several light probes can be placed in a scene; the material’s light picker will assign weights to each to determine the importance in the lighting step based on the rendered object’s position relative to each.

You can use normal lights alongside the light probes, and in case of the default material system, you can specify which lighting component (diffuse and specular) is affected by which light types (material.diffuseLightSources and material.specularLightSources accepting LightSources.LIGHTS, LightSources.PROBES,  or LightSources.ALL). This can be useful if you want to use light probes for diffuse global lighting, but want specular lights from traditional light sources without those affecting the diffuse light.

Pre-process step

A downside of this approach over conventional light sources is that it requires a pre-process step to generate the cube maps. To start with, you need to generate (HDR) cube map renders for every point where you’ll want to put a light probe. From this, two process steps can be applied: a diffuse convolution and a specular convolution, depending on which type of lighting the probe needs to support. These can be generated in the eminent Paul Debevec’s HDRShop. I’ll warn you, however, that these convolutions can be very time-consuming – but the results are very accurate. If you’re willing to sacrifice some accuracy and get your hands dirty, you can write your own “convolutor”. Be warned however, that Flash doesn’t support HDR in BitmapData or textures, so if you’re using BitmapData and Stage3D, you’ll end up with under-lit results due to the low dynamic range of “bright” light. Having said that, I’ll quickly outline the principles behind it, just in case it interests anyone.

Lighting functions are spherical functions (the domain is the unit sphere, being mapped to scalar values), so you’ll need to integrate the diffuse or specular lighting function over the unit sphere of surface normals/view reflections. In fact, integrating over the hemisphere is usually enough since there’s no light coming from inside objects in our case. But as I was lazy and it was much easier to just implement it for the entire sphere I’ll use that approach For basic (Lambertian) diffuse reflections, this semi-formally boils down to:

Where N is the surface normal, “ω_i” is the incident light vector, and L(ω_i) is the light colour coming from the direction ω_i. The part “max(N.ω_i, 0) L(ω_i)”, is in fact just the same as the diffuse lighting calculation done for point and directional lights (where L is a constant function). In this case, L(ω_i) is simply a sample from the HDR cube map we rendered before. The integral just means that we’re taking the diffuse contributions for all directions (ie: all vectors ω_i in the unit sphere S) and accumulating them. Finally, to be able to sample the incoming light for each normal in an environment map, we would need to perform this calculation for every point in that map (remember, cube maps coordinates are simply 3D vectors, in this case representing the surface normal N).

In theory, the integral means we’d be summing up infinite amounts of infinitesimal samples. In practice, we’ll be using discrete steps to solve this. That’s okay, since we only have finite amount of samples to work with anyway; 6*n² pixels to be exact, n being the size of the HDR cube map. So, the most accurate way of doing things would be to add up all the contributions from the source map, and do that for each pixel in the destination map. If we’re rendering to a cube map of size m, that means it’s an order of 6*m² * 6*n² = 36m²n² (6 sides of m*m, each pixel sampling 6 sides of n*n). This can easily become a very expensive operation. Luckily, there are some numerical methods to solve this faster (at the cost of accuracy, or what did you expect). The most useful in this case is definitely Monte Carlo integration, which allows less samples from the source material per pixel. There are many articles written about the subject, and a great introduction is found in this article by Robin Green.

Enough to get you cracking, I think

Demo

See the Cornell Boxness of it all!

Use the arrow keys to move the head, space bar to toggle the texture (shows the lighting impact), and finally click and drag to rotate the head.

The head model and textures are still by Lee Perry-Smith (who by now must feel pretty awkward appearing in random tech demos). Source for the demo can be found in the texture-refactor branch of the examples repo (which will be merged to master together with the dev branch).

The environment maps were made as outlined before using a custom generation tool, and placed close to each corner of the box where they were generated.  I was using Stage3D (and thus a low dynamic range) to generate them, so I had to do some touch-ups in Photoshop to get them to look right. I’m not done investigating the options for further development of such map generation tools, but who knows they’ll make it public some day. In any case, HDRShop is more reliable! Diffuse lighting is set to use light probes only, while specular lighting uses the point light only.

The demo also shows how to use point light shadows, rim lighting and light mapping for added static shading.

If you’ve been following the updates on the Away3D blog, you’ll have noticed the updated roadmap mentioning the dev branch on Github. This branch is basically where we’re working to get things towards Beta, so take a look if you’re curious to see where we’re headed. I personally advise anyone to use this branch if you’re starting out with a new project but make a snapshot of the revision you’re working with. Some things will still change and might very well break your code

We’ll put up some proper blog posts introducing new features as we get to Beta, but for now I’ll quickly explain what I’ve personally been working on, and how I broke your existing code I’ve also went ahead and updated the source code for the demos in previous blog posts to match the dev branch, so you can see how it differs concretely.

Death to BitmapMaterial! All hail TextureMaterial!

One of the biggest changes, I think. Favouring composition over inheritance, we deprecated BitmapMaterial, VideoMaterial, MovieMaterial etc. They have been replaced by a single TextureMaterial, which accepts a Texture2DBase object. In practice, this will for instance be a BitmapTexture (VideoTexture etc are still in development). This way, when supporting new texture types you don’t need to extend every type of material that needs to extend this material type, which is especially a mess when you start creating custom materials (and need to extend/duplicate those to support all possible texture types). The same applies for cube maps (fe: BitmapCubeTexture), the normal and specular maps that are set on the default materials, and the view’s backgroundImage (now simply View3D::background). It’s a big change that might annoy many of you, but it’s necessary. With things like ATF support coming up (and who knows what else), things would simply become unmanageable and error-prone otherwise.

Furthermore, there are some “special” convenience textures that can be used for the default materials. For instance, these materials expect a specular map to have the specular strength on the red channel, and the specular power (gloss) on the green. SpecularBitmapTexture allows you to pass two greyscale BitmapData objects (gloss is optional) and composites the data correctly. When using texture splatting, you can use SplatBlendBitmapTexture, which takes 3 BitmapData objects containing the blending value for each splatting layer and places each on the red, blue and green channels as expected by TerrainDiffuseMethod. However, in both cases, it’s usually better to do these things in your offline asset preparation stage and use simpler texture objects instead.

Light Picking

Another rather big change for those working with lights, but one that offers great benefits. Instead of assigning the lights as a static array as before, materials now expect a LightPickerBase object. Right now, that means StaticLightPicker, which in turn accepts the array of lights. Why this extra object? This way, you can create a DynamicLightPicker object that selects the most important lights from the EntityCollector, allowing different lights to be used without reassigning the light array (and as a result potentially invalidating the material). As light collection is often game-engine dependent, providing our own dynamic light picker is not the highest priority. At least it’s good to know the possibility is there, right?

Disposal

Before, most dispose(…) methods accepted a “deep” parameter, which caused “sub-assets” (Geometry of a Mesh, BitmapData of a BitmapTexture, etc) to be disposed as well. While in some cases this saves you from writing a trivial few lines of disposal code, in many cases it just is an open door to invalid state (accidentally disposing a BitmapData used by another BitmapTexture) and ambiguity (what will be cleared, and what will remain uncleaned?). Allowing an object to destroy objects it did not create is poor resource management and generally bad practice in my opinion. Hence the new credo: you clean up what you create!

Ambient lighting

There have been some changes in how ambient lighting works. Before, the ambient and ambientColor of the materials dictated how much ambient light is coming from the lights, rather than how much ambient light is actually reflected by the material. That has been changed, and lights now have ambient and ambientColor properties as well. As a result, ambient light now functions similar to diffuse and specular light. The lights emit it, the material reflects it. It’s usually best to only set ambient light properties on DirectionalLight objects, since ambient light is global. PointLight objects can get culled if they’re not affecting anything in the frustum and their ambient contribution would be dropped as a result.

Other changes and new features

• Big refactor on how the materials and animations are compiled to facilitate building custom materials
• Slight restructuring of the render flow to allow more intricate custom renderers
• Light probes (default materials only support convoluted cube maps at this point)
• Reduced shadow swimming with shadow mapping
• RefractionEnvMapMethod for environment-map based refraction mapping
• AnisotropicSpecularMethod for anisotropic specular highlights
• Shadow mapping for point lights (HardShadowMapMethod only)
• AlphaMaskMethod to influence the material’s alpha, optionally based on a secondary UV set
• Light map methods: DiffuseLightMapMethod to apply light maps as diffuse light, and LightMapMethod to apply it on the final shaded result
• WrapDiffuseMethod to perform a very cheap fake way for subsurface scattering, as outlined in GPU Gems #1

I hope to scrape together some time to write some things about these new features as soon as possible. A recurring promise, I know, but bear with me here

Oh such busy times! I’m not keeping up with my planned blog posts as much as I’d like, but between work and Away3D 4.0 updates there’s not as much time left to actually write about it. But, I did want to share a video from my presentation on Flash on the Beach 2011, showcasing some of the things that can be done relatively easily with Away3D 4.0 by compositing several shadow methods and so on. It also showcases some tricks that aren’t in the engine yet, but will be after some refactoring. It’s very much a coder’s demo, so don’t expect too much artistic sense

Some of the things shown that I’d like the point out:

• Subsurface scattering for ice and gelatinous materials
• Large amount of independently moving scene graph objects (1000 cubes)
• Terrain texture splatting
• Partial shadow mapping for large view frustra (coming soon)
• Dynamic cube map refraction + colour aberration (coming soon)
• Dynamic cube map reflections (coming soon)

I’ll elaborate on some more once they’re available in the engine, I promise!

The Away3D team also recently joined forces with Evoflash (in particular Simo) again to create a real scene demo for Adobe MAX. Read all about that one Simo’s blog. Great job guys!

Time to dig deeper into Away3D 4.0 and see how you can structure and manage your content to get the best performance! If you haven’t read the previous posts, it wouldn’t hurt to do so now, since this one will be all about Materials (on the Mesh side of things) and Geometries.

In any sort of project, if you have data collections that use a lot of memory, it makes sense not to duplicate them if you want to share the same data across several clients (clients simply meaning “users of the data”). Instead, you make all clients refer to the same data object, or model. Similarly, you probably wouldn’t want to perform the same expensive operations on this data for every client. The shared model can take care of that only once for all clients. In the Away3D scene graph structure, the Geometry is essentially your model, and a Mesh refers to it. Many Meshes can make use of the same Geometry. Similarly, Materials can be shared across Meshes as well. This all sounds very logical, yet for some reason I can’t fathom, this is a principle that’s often violated against. Even though Away3D internally tries to minimize memory use and tries to share things as best as it can (it caches Program3D instances, Texture objects, etc., across material instances), I can’t stress this enough: reuse Material and Geometry instances whenever possible. Not only does it limit CPU and GPU memory usage, it’s also essential for performance. I’ll explain why.

Reusing Materials

Every material uses a “shader program”, which is represented in Actionscript by a Program3D object. It’s a combination of a vertex shader and a fragment shader. I won’t go into the subject of shader programs in detail, but if you’re interested, check wikipedia for starters. A large task of the shader program is to define the colour of every pixel that’s being drawn. In other words, it controls how your rendered object will be coloured, textured, lit, shaded, … If it’s on your screen, it passed through a shader program. As a result, a different material means it needs a different Program3D. When drawing objects, the gpu needs to be told to use a different program every time for every material. Unfortunately, this switch is typically very expensive and something you’d want to prevent as much as possible. To this end, Away3D sorts all the objects that need to be rendered (the so called “renderables”) primarily according to their materials so all objects with the same material will be rendered consecutively. If you’re using different material instances for things that look exactly the same, it means more programs need to be switched needlessly.

Sharing materials allows more renderables (here SubGeometry objects) to be rendered with less switches

Okay, I’ll admit, that isn’t entirely true. For those interested, I’ll explain what really happens. Whenever a material is created or changed so that it needs to create a new shader program, the vertex and fragment code is regenerated. A Program3D is requested from a central cache, but if a Program3D object already exists for that exact code combination, the existing object is returned. Different materials using the same code also use the same programs, which means there won’t be any program switching. However, it’s easy enough to accidentally have small differences between materials so that the code – and in turn the program instance – changes. And the following is still true regardless:

Every time a material changes, certain data needs to be uploaded to the GPU: Textures, lighting properties, … No caching of Program3D objects is going to prevent that. If you use a new material instance for every object, these uploads may occur for every object. Share a material, and it only happens once for all the renderables using it.

Note: if you really need several material instances, but you’re using the same BitmapData objects as textures or normal maps, at least try to share those across materials unless you have a very good reason not to. RAM’s not for wasting! Furthermore, there’s a limit to the amount of Texture objects (a buffer object representing the image data on the GPU) that can be created, and for every BitmapData, a Texture is created. You wouldn’t want to go over the limit!

Reusing Geometries

Similarly, Meshes that use the same data should all refer to a single Geometry instance. A very common example is a game in which there are several occurences of the same monster type spread throughout the game map. You can have one “beer demon” Geometry, and place many beer demons in your level by making the different Mesh objects refer to that single Geometry instance. Not only does this use much less memory, it also causes less instances of VertexBuffer3D to be created (these are buffers representing the vertex data on the GPU side of things). There’s a limit on how many can exist at any given time and if you exceed it, an error will be thrown and your game will crash. Sharing Geometry objects can also result in better performance in some cases, because if the same Geometry was used for the previously rendered object, it may not have to reupload its data to the GPU!

Material/Geometry Tandems

In many cases, especially in games, you’ll have a collection of game object “types”, such as different monsters, a barrel, weapon/health/power-up pick-ups. Each of these objects will probably have their own specific Geometry and Material objects. Makes sense, since you typically wouldn’t want to use a barrel geometry for a monster, or a shotgun texture on a health boost pick-up, would you? This one-on-one mapping is a pretty fortunate situation, because it means that the rendering for these objects can be batched very efficiently. Only when different objects will be rendered, does any material or geometry data need to be uploaded, with the exception of scene-graph-specific data such as transformation matrices.

Rendering objects with one-on-one Material/Geometry combinations

Making sure that the same Geometry and Material instances are used across all instances for the same game object type is easy. The Mesh class contains a clone() method that creates a new Mesh instance that refers to the same Geometry and Material. This way, you can build a library of reference Meshes that are not actually added to the scene. Instead, you populate your scene with these clone objects. You can dispose of them whenever they’re picked up (health pack) or killed (monster), while the reference mesh and its material and geometry can be disposed when everything is unloaded. For this reference library, you can use Away3D’s AssetLibrary, which will take care of all your loading and management needs

For example

Here’s a comparison:

Demo NOT reusing anything (warning, can be very slow!)

Demo reusing both Geometry and Materials

Source for both

I’ll admit, a large cost in the first demo is the creation of the objects rather than their use. This creation is a constant cost, however, and you can see the performance drop drastically over a short time. So… Enough incentive?

Update

In the “dev” branch (and when we go Beta it will also be in the master branch), BitmapMaterial was removed in favour of TextureMaterial, accepting a texture rather than a BitmapData. It goes without saying that you should share these texture objects whenever possible rather than creating new textures with the same content.

All sources for this post have been updated to use said dev branch: https://github.com/away3d/away3d-core-fp11/tree/dev

It’s time to elaborate on the previous blog post’s topic and investigate one of the most important scene graph nodes: the Mesh. It’ll probably be the object you’ll be working with most, and it allows a form of flexibility that can really provide a benefit to your memory usage and performance. That is, of course, if you know how. I’ll talk more about performance and memory usage next time, for now it’s just important to understand how Mesh works.

Meshes and Geometries

Meshes and Geometry objects are closely related. Geometries provide the – you guessed it – geometry of a 3D object: the vertices, triangle indices, UV coordinates, vertex normals, and what have you. Mesh objects are there to give a Geometry a place in the scene graph (and as such, a position in 3D space), as well as link it to a material. Why bother with the distinction, you may wonder. Why not just place the geometry data inside the Mesh class and be done with it? In many cases, you’d want to use the same geometry in several places. Different monsters of the same type throughout a level, for instance. The Geometry object allows you to save memory and limit the amount of data that’s uploaded to the GPU.

Both Meshes and Geometry mirror each other in that they both contain “sub”-objects (SubMesh and SubGeometry, respectively). Each SubMesh object links to a corresponding SubGeometry object in the Mesh’s Geometry object. Each SubGeometry provides the vertex and triangle data for each part that will make up the entire object. Confused? Check the graph below. The reason for this subdivision is mainly so multiple materials can be used within a single Mesh. Because of how the GPU issues draw commands, materials cannot be assigned to individual triangles in the model. Instead, several SubGeometries are used, and each can be rendered with a different material. This is achieved by assigning a material to the corresponding SubMesh. SubMesh materials are optional; if not provided, the main Mesh object’s material is used for rendering. In other words, the SubMesh material property overrides the Mesh material property for the corresponding SubGeometry.

Time for a practical example. A car Mesh can refer to a Geometry object containing a SubGeometry for the car body and one for the windows. This way, a metallic BitmapMaterial can be used for the car body, and a transparent ColorMaterial can be used for the windows, by assigning the materials to the SubMesh objects.

The scene graph inheritance structure (black arrows) and the Mesh/Geometry architecture (white arrows)

The Scene Graph in Action

If you’re still a bit confused even after looking at the magnificent chart above (yes, that is the same one as in the previous post – sue me), a demo will definitely help. And what way to better illustrate a hierarchical scene graph than one illustrating a blatant lack of inspiration as well: a mini solar system! Not everything in the source is as pragmatic as it could be, but as it’s done for purely illustrative purposes, I’ll forgive myself just this once. Play around with the source, and things should start making more sense.

Demo

Source

And that’s it for now! Any questions? shoot!

Update

All sources for this post have been updated to use the dev branch of Away3D 4.0: https://github.com/away3d/away3d-core-fp11/tree/dev

I’m planning some future blog posts dealing with tips to get better performance out of your Away3D 4.0 (FP11) projects. When working with the GPU, it’s easy to get unexpected slowdowns due to inefficient content. There are a lot of common mistakes and misconceptions that I see popping up time and time again, and I reckon it’s a good idea to try and rid the world of them. Before that, it’s important to understand one of the most central aspects of the engine: the scene graph. It forms the basis of many of the performance tips that will follow, so despite the seemingly trivial nature of the subject, it is important to explain things in proper detail.

The scene graph is essentially the 3D equivalent of the Flash display list. If you’ve worked with Away3D before, or any other 3D engine for that matter, it’s nothing new; every version of Away3D has had a scene graph, even if it was never been called that way, and it’s simply the Scene3D object and all its children: ObjectContainer3D, Mesh, Sprite3D, … It’s the building blocks you use to construct your 3D scene in a hierarchical way that feels natural. There are some differences with older versions of the engine, especially how Mesh objects are constructed, so let’s step through the different objects.

Scene graph objects

The scene graph is made up of objects (nodes) that form a tree data structure. Nodes can be added as a child to other nodes, which makes their transformation (position, rotation and scale) relative to that of their parents’.

• Scene3D: Essentially, this is the root node of a scene graph. It wraps a root ObjectContainer3D and links it to the spatial partitioning system (more on that in a future post). Whenever a View3D is created, an empty Scene3D is created by default if one is not passed along. Since it’s a root node by design, a Scene3D object cannot be added as a child to other containers.
• ObjectContainer3D: Comparable to Flash’s DisplayObjectContainer, it is the most basic scene graph node. It can be added as a child to other ObjectContainer3Ds (or subclasses), and in turn it can be a parent of other ObjectContainer3D (or subclass) objects. Despite the name, child nodes aren’t necessarily “contained” by the container in the most literal sense. It merely implies that the child objects’ transformation is relative to that of the parent, much like Flash Sprite objects that are a child of a DisplayObjectContainer. Move or rotate the parent, and the child will move or rotate along. For instance, a rider object can be a child of a horse object so that when the horse moves, the rider will always be in the correct place without having to explicitly update its position. It does not mean – assuming your game is within the bounds of the morally acceptable – that the rider is inside the horse. This may seem like a distinction too trivial to even point out, but it’s a misconception that has managed to confuse a good few people in the past. The results are often wastefully constructed scene graphs and superfluous transformation updates, simply because of this misinterpretation. Every scene graph object that you manipulate to generate your scene (that excludes Scene3D itself), extends ObjectContainer3D.
• Entity: While it’s an abstract class and you should never use it directly, it’s important enough to know of its existence, especially if you’re looking into creating your own scene graph objects. It forms the base for any object that has a “physical presence”. Entities are collected in the render loop, and are dealt with differently depending on the concrete subclass.
• Mesh: The most typical renderable 3D object, consisting out of a Geometry that defines vertices, triangles, and all that jazz. It also provides the material with which the object is rendered. Mesh objects will be the central subject of the next blog post, so I’ll keep it brief here.
• Sprite3D: A so-called “billboard”, basically a 2-triangle plane that always faces the camera. Remember the monsters in old shooters like Doom or Duke Nukem 3D? Sprites, baby!
• SkyBox: A special case scene graph object, used for faking the sky or environment in a scene. As it’s always supposed to be “as far away as possible”, it can’t be moved and will always center itself around the camera and size itself so it’ll fit within the camera frustum. There’s not supposed to be more than one SkyBox in the scene.
• PointLight/DirectionalLight: These are light objects that can be used for material shading and casting shadows (DirectionalLight only). Traditionally, lights weren’t part of the scene graph in Away3D, but now they can be added in order to make lights dependent on other objects’ position (fe: the light on a miner’s hat). However, because directional lights are “infinitely far away” to approximate very distance light sources such as the sun, their position is in fact irrelevant. It’s recommended to add them to the scene directly. However, the position in combination with lookAt could be used to create sun cycles, but yeah let’s not go there now
• Camera3D: Cameras are used to render the scene to a view, and don’t necessarily need to be added to the scene graph (by default, they aren’t). However, it can be convenient if you want to make your camera’s position and orientation depend on another object (a rigid chase camera for instance), which is also a change from older versions of the engine.

The scene graph inheritance structure (black arrows) and the Mesh/Geometry architecture (white arrows)

See, not that different from what you probably already know! However, the way Meshes are constructed, and how they relate to Geometry objects has changed considerably, and can be an important aspect when creating efficient content. And that’s… for the next blog post!

Molehill, Adobe’s new API to leverage the GPU for rendering, was made public some time ago in the form of an “incubator” Flash Player release, and along with it we released an early pre-alpha version of Away3D 4.0, codenamed “Broomstick”. I’ve been working on the new engine for quite some time now, and since the release I hadn’t had the time to post any more information aside from what was shown on the blog post. Time to change that, I’d say!
Before you do anything else, you’ll need to install the Incubator Flash Player, because the demos will be using that. Secondly, there’s no guarantee that anything related to Away3D 4.0 will be exactly the same in the final release, as it is of course an early alpha. Finally, source for all demos is in the usual Away3D repository. They’re a bit messy, since they were actually just quick test files to see if everything still works

Subsurface scattering

Subsurface scattering is basically what happens to light when it travels through a translucent surface. Instead of either simply reflecting off a surface or passing through it, the light will enter the material, reflect a good couple of times before exiting the surface again and reaching the eye. This behaviour causes a general blurring of the diffuse lighting on an object and allows light to shimmer through less “thick” regions. Just think about how a candle’s flame shines through the wax.

There’s a few rendering techniques that perform subsurface scattering to some extent. A popular approach is to render the diffuse lighting to a texture and simply blur it slightly. However, I find that this approach doesn’t allow light to travel through an object and so I opted for another approach: using depth maps. Similar to shadow mapping, a depth map is rendered from the point of view of the light, and when rendering, we compare depths of the rendered pixel to the value in the depth map. Instead of using this to determine whether a pixel is in shadow or not, we can use the difference between these values to determine the “thickness” of the object, or how far light has to travel through it to be able to reach the eye (see my amateurish schematic on the right ).

Depending on this distance, we can colour the pixel differently. We define a colour the light gets by traveling through the object (this depends on the actual material), and the final pixel’s colour ranges from this “scatter colour” to the material’s (texture) colour, depending on how thick it is.

This technique can be used to make materials look like wax, such as in this demo.

Advanced Skin Rendering

Skin is in fact a translucent object, so we can use subsurface scattering to increase the realism, as usual rendering techniques often create a harsh or clay-like effect. Instead, subsurface scattering softens the general shading and allows light to travel through more translucent regions such as the ears, softer parts of the nose, etc.
To further increase the realism, the specular highlights are modulated using a fresnel term. As with most objects in reality, the reflectivity of an object changes depending on the view angle, and with it the strength of the highlights. When observing skin head-on, there’s usually not a strong direct light reflection, but it can usually be perceived on shallow angles.
The result of both can be seen here.

In Away3D 4.0

First of all, to get started using Broomstick, check out this article by John Lindquist. He did a great job explaining how to get started!
Default materials in Away3D 4.0 are currently implemented using “methods”, which are properties of a material that define how the shaders work in a modular fashion. This way, there’s no need for a billion variation of Material classes for each combination. For subsurface scattering, we need to change the diffuseMethod property of a material and assign an instance of SubsurfaceScatteringDiffuseMethod. For the fresnel specular highlights, we assign an instance of FresnelSpecularMethod to the specularMethod property. It’s all in the source, so knowing this, you should be able to make sense of what’s going on in there

Demos

In case you skipped through the rest of the article, here’s a demo recap. Loading times may be long, as the models are pretty big

Wax Statue : Illustrating subsurface scattering to generate a material resembling wax.
Head rendering: Using subsurface scattering and a specular fresnel term to increase realism in skin rendering. Press any key to toggle the subsurface scattering, so you can see the difference. Note the light shining through the ears, for example.

Since I’ve been working on Away3D 4.0 (Broomstick) for the past few months, it was pretty hard to find the energy to keep this blog up to date, and the promised part 2 of Hacking the Lite kept getting delayed. And delayed. And then delayed some more. Having access to the GPU also caused some debate on whether this series would still be relevant enough. However, after looking at my initial goals for it, I decided it is. After all, the idea was to inspire people to start playing around, and to get people to think in terms of using shaders. Most of the concepts outlined here can be used in Molehill as well. And of course, it’s still some time before we can use the GPU in actual commercial projects So on with the show! This part of the series will build heavily upon the previous part, so it’s perhaps a good idea to review that again.

Shadow maps

Casting shadows is often done using shadow maps. In this case, we’d render a “depth map” of the scene from the light’s point of view. This way, we have a texture containing the distances to the objects closest to the light for every projection ray (which is a single pixel in the shadow map). When rendering the scene, we project every point we are rendering to that shadow map. We can then compare the depth value of the point-to-be-rendered with the value in the shadow map. If it is further away from the light, it must be in shadow and should not be lit. This approach works well, but for current versions of Flash, it’s not very feasible. But, since we’re hacking, there’s other options!

Augmenting projective texture mapping

If you take a look at the schematic on the left (click to enlarge), showing where a shadow should fall, you can see it’s much like projection mapping. The sphere is being perspectively projected unto the floor plane with the light as the projection’s origin. This is the exact same projection that projects the light texture onto the objects. As you can see, if we project the sphere onto the light’s texture first, and project that in turn to the floor, the shadow region remains the same. As a result, we could actually simply project and draw the sphere into the light map before it is applied to the floor.

Order and form

To render a scene with multiple objects, we have to take care to render things in correct order. We need to draw the objects closest to the light first, because they will obviously be casting shadows on object further from the light. Starting out with a clean light map (ie.containing no shadows), we project it on the closest object and then draw that object’s shadow into the light map. Move on to the second closest object, light it, and draw the shadow again. And so on until the last object was reached.

That works great for convex objects, as long as they don’t intersect. For concave or intersecting convex objects, it’s possible that object A casts shadows on object B as well as the other way around. Since we’re rendering everything in a strict order, that’s simply not a possibility. However, this error should generally be acceptable. Another problem is self-shadowing for concave objects, which is impossible using this “cheat”.

Demo and source

Fake shadow mapping demo: really just an adapted version of the previous one.

The source is again on Google Code!

For those living near Cologne, Germany, I’m stoked to announce I’ll be speaking at FFK (FlashForum Konferenz) this year. I’m particularly excited, as last year’s edition is still one of my best experiences in 2010, even though a certain Eyjafjallajökull caused an interesting aftermath for many . Titled “Keeping it Real”, the session will mainly revolve around mimicking reality using real-time computer graphics. So expect a lot of experiments that’ll hopefully show that beauty’s perhaps not always in the entirely abstract!

What’s more, apart from a session, my good friend Frank Reitberger and I will be co-hosting a workshop “Das Efx” on Thursday. It’ll be all about playing around with graphics and 3D rendering, ranging from relative basics to more advanced topics. So if you’re into this stuff, but don’t really know how to get started, be sure to come!

Finally, the next part on Hacking The Light is still in the works. Sadly, there’s a lot that keeps me distracted in the meanwhile, so hang in there!

Hope to see you at FFK!

Exciting times afoot! In the not too distance future, we’ll finally get what we’ve been nagging about! I’m of course talking about Molehill, the new hardware-accelerated API Adobe is working on. If you’re curious what we’re doing with Away3D, take a peak at the demo we did with the guys from Evoflash for Adobe MAX. Old news, I know. Moving on.

All that and more is still some time in the future, and in the meanwhile we need to make do with what we currently have available. As part of a session at AUG XL and FlashPlatform, I gave some examples of what we currently can do with Away3D Lite if we’re willing to get our hands a bit dirty. As examples I explained some effects I did for the Spiral Out demo. Now, it’s time to clean up the code mess and explain a thing or two to a slightly broader audience

Basically, what I’ll do is show some things that involve making assumptions that prevent the code from being scalable (ahhh… you got me… that’s just a nice way of saying I’m cheating!). The first step isn’t actually cheating at all, and it’s an old and pretty standard shading technique. However, we’ll need this for part 2 in which we’ll do nothing but cheating, and since no one ever seems to do shaders in Lite I thought it’d be a good way to get started.

This it what we’ll end up with. Not the most attractive demo, but it’ll do. Lengthy explanations ahead, for which the source can come in handy (under examples/projectionMapping)

Projective Texture Mapping

Projective texture mapping, as the name clearly suggests, is a texture mapping technique that mimics a light projecting a texture on objects in the scene (think of a projector slide, a lamp shade’s projections on the walls, …). As I said, there’s nothing new or special about this; it’s been done countless times before. The picture on the right pretty much shows all there’s to it (click to enlarge).

Essentially, we place the “light texture” in front of the light. All we need to do, is draw a line from the position we’re currently shading (the “target point”) to the light (the “source point”) and figure out where that line intersects the plane. Grab the light texture’s colour at that position and multiply it with the object’s base texture colour to get the final shaded colour. We can also incorporate additional shading calculations, particularly normal mapping.

Exactly how we calculate the intersection is up to you. You can do a line-plane intersection calculation, or you can – surprise surprise – use a perspective projection matrix to project the target point onto the texture. So rather than projecting the texture onto the object, we’re effectively projecting the object on the texture first. The latter approach is much more flexible and intuitive, since it’s exactly how a camera projection works (in part 2, we’ll see that this will come in handy too).

Shading in Away3DLite

Away3DLite, minimal as it is, has only very limited support for shading. There’s a Dot3BitmapMaterial that no one seems to know about, though, courtesy of Mr. Bateman I recommend taking a look at that code, just to see how you could implement a custom shader. At a minimum, all you’d need to do is extend BitmapMaterial and override the updateMaterial method (don’t forget to include and use the namespace arcane). In there, you can do anything you like as long as you end up passing a valid BitmapData object to the _graphicsBitmapFill.bitmapData property. For shading purposes, it’s best to keep a second BitmapData object to which we’ll be rendering the shaded version of the texture, leaving the original _bitmap object intact for subsequent frames. Of course, for any advanced shading on BitmapData objects, Pixel Bender is king, so we can run a ShaderJob in there.

Taking a look at the source for the ProjectiveTextureMaterial, that’s pretty much exactly what is happening, with one difference. We’re exposing a manual update method rather than overriding updateMaterial. This seems rather odd (and in this particular case it is) but we’ll need that next time (remember, we’re hacking anyway)

There’s of course a final caveat, in that we don’t have any position information since we just have a bunch of BitmapData objects, no actual geometry. And that is of course something essential if we want to do any projecting at all. For that we can take the same approach as the Away3D Pixel Bender shaders do: position maps. Whereas a normal map encodes the normal for every texel, a position map encodes the object position at that point. In Away3D, the pixel shaders generate a position map automatically from the geometry when a material is created, but in Lite, we’ll settle for creating them manually. This actually makes it possible to add details from a height map, providing the shader with much more detailed topological information than what would have been generated from a bunch of triangles. As a result, the projected map can be subtly displaced as it would on a real uneven surface. It’s not very hard to create position maps procedurally (which is part 3) if you keep in mind that xyz maps to rgb and 0×00-0xff for each colour channel maps to the minimum and maximum bounds of the respective coordinate (so where x == minX, r would be 0).

Demo and source

To see the effect in action, check out the demo. Move the mouse to move the projector, click and drag to move the camera, and use the mouse wheel to move the projector closer or farther from the scene. There’s also a small projection avatar floating around that shows to position and orientation of the projector.

Source can be found in my repo at Google Code.

A long post, which I hope is useful for people. If not, let me know and I’ll keep it more concise next time!
And oh yes, happy New Year’s!