Since I’ve been working on Away3D 4.0 (Broomstick) for the past few months, it was pretty hard to find the energy to keep this blog up to date, and the promised part 2 of Hacking the Lite kept getting delayed. And delayed. And then delayed some more. Having access to the GPU also caused some debate on whether this series would still be relevant enough. However, after looking at my initial goals for it, I decided it is. After all, the idea was to inspire people to start playing around, and to get people to think in terms of using shaders. Most of the concepts outlined here can be used in Molehill as well. And of course, it’s still some time before we can use the GPU in actual commercial projects So on with the show! This part of the series will build heavily upon the previous part, so it’s perhaps a good idea to review that again.

Shadow maps

Casting shadows is often done using shadow maps. In this case, we’d render a “depth map” of the scene from the light’s point of view. This way, we have a texture containing the distances to the objects closest to the light for every projection ray (which is a single pixel in the shadow map). When rendering the scene, we project every point we are rendering to that shadow map. We can then compare the depth value of the point-to-be-rendered with the value in the shadow map. If it is further away from the light, it must be in shadow and should not be lit. This approach works well, but for current versions of Flash, it’s not very feasible. But, since we’re hacking, there’s other options!

Augmenting projective texture mapping

If you take a look at the schematic on the left (click to enlarge), showing where a shadow should fall, you can see it’s much like projection mapping. The sphere is being perspectively projected unto the floor plane with the light as the projection’s origin. This is the exact same projection that projects the light texture onto the objects. As you can see, if we project the sphere onto the light’s texture first, and project that in turn to the floor, the shadow region remains the same. As a result, we could actually simply project and draw the sphere into the light map before it is applied to the floor.

Order and form

To render a scene with multiple objects, we have to take care to render things in correct order. We need to draw the objects closest to the light first, because they will obviously be casting shadows on object further from the light. Starting out with a clean light map (ie.containing no shadows), we project it on the closest object and then draw that object’s shadow into the light map. Move on to the second closest object, light it, and draw the shadow again. And so on until the last object was reached.

That works great for convex objects, as long as they don’t intersect. For concave or intersecting convex objects, it’s possible that object A casts shadows on object B as well as the other way around. Since we’re rendering everything in a strict order, that’s simply not a possibility. However, this error should generally be acceptable. Another problem is self-shadowing for concave objects, which is impossible using this “cheat”.

Demo and source

Fake shadow mapping demo: really just an adapted version of the previous one.

The source is again on Google Code!

Exciting times afoot! In the not too distance future, we’ll finally get what we’ve been nagging about! I’m of course talking about Molehill, the new hardware-accelerated API Adobe is working on. If you’re curious what we’re doing with Away3D, take a peak at the demo we did with the guys from Evoflash for Adobe MAX. Old news, I know. Moving on.

All that and more is still some time in the future, and in the meanwhile we need to make do with what we currently have available. As part of a session at AUG XL and FlashPlatform, I gave some examples of what we currently can do with Away3D Lite if we’re willing to get our hands a bit dirty. As examples I explained some effects I did for the Spiral Out demo. Now, it’s time to clean up the code mess and explain a thing or two to a slightly broader audience

Basically, what I’ll do is show some things that involve making assumptions that prevent the code from being scalable (ahhh… you got me… that’s just a nice way of saying I’m cheating!). The first step isn’t actually cheating at all, and it’s an old and pretty standard shading technique. However, we’ll need this for part 2 in which we’ll do nothing but cheating, and since no one ever seems to do shaders in Lite I thought it’d be a good way to get started.

This it what we’ll end up with. Not the most attractive demo, but it’ll do. Lengthy explanations ahead, for which the source can come in handy (under examples/projectionMapping)

Projective Texture Mapping

Projective texture mapping, as the name clearly suggests, is a texture mapping technique that mimics a light projecting a texture on objects in the scene (think of a projector slide, a lamp shade’s projections on the walls, …). As I said, there’s nothing new or special about this; it’s been done countless times before. The picture on the right pretty much shows all there’s to it (click to enlarge).

Essentially, we place the “light texture” in front of the light. All we need to do, is draw a line from the position we’re currently shading (the “target point”) to the light (the “source point”) and figure out where that line intersects the plane. Grab the light texture’s colour at that position and multiply it with the object’s base texture colour to get the final shaded colour. We can also incorporate additional shading calculations, particularly normal mapping.

Exactly how we calculate the intersection is up to you. You can do a line-plane intersection calculation, or you can – surprise surprise – use a perspective projection matrix to project the target point onto the texture. So rather than projecting the texture onto the object, we’re effectively projecting the object on the texture first. The latter approach is much more flexible and intuitive, since it’s exactly how a camera projection works (in part 2, we’ll see that this will come in handy too).

Shading in Away3DLite

Away3DLite, minimal as it is, has only very limited support for shading. There’s a Dot3BitmapMaterial that no one seems to know about, though, courtesy of Mr. Bateman I recommend taking a look at that code, just to see how you could implement a custom shader. At a minimum, all you’d need to do is extend BitmapMaterial and override the updateMaterial method (don’t forget to include and use the namespace arcane). In there, you can do anything you like as long as you end up passing a valid BitmapData object to the _graphicsBitmapFill.bitmapData property. For shading purposes, it’s best to keep a second BitmapData object to which we’ll be rendering the shaded version of the texture, leaving the original _bitmap object intact for subsequent frames. Of course, for any advanced shading on BitmapData objects, Pixel Bender is king, so we can run a ShaderJob in there.

Taking a look at the source for the ProjectiveTextureMaterial, that’s pretty much exactly what is happening, with one difference. We’re exposing a manual update method rather than overriding updateMaterial. This seems rather odd (and in this particular case it is) but we’ll need that next time (remember, we’re hacking anyway)

There’s of course a final caveat, in that we don’t have any position information since we just have a bunch of BitmapData objects, no actual geometry. And that is of course something essential if we want to do any projecting at all. For that we can take the same approach as the Away3D Pixel Bender shaders do: position maps. Whereas a normal map encodes the normal for every texel, a position map encodes the object position at that point. In Away3D, the pixel shaders generate a position map automatically from the geometry when a material is created, but in Lite, we’ll settle for creating them manually. This actually makes it possible to add details from a height map, providing the shader with much more detailed topological information than what would have been generated from a bunch of triangles. As a result, the projected map can be subtly displaced as it would on a real uneven surface. It’s not very hard to create position maps procedurally (which is part 3) if you keep in mind that xyz maps to rgb and 0×00-0xff for each colour channel maps to the minimum and maximum bounds of the respective coordinate (so where x == minX, r would be 0).

Demo and source

To see the effect in action, check out the demo. Move the mouse to move the projector, click and drag to move the camera, and use the mouse wheel to move the projector closer or farther from the scene. There’s also a small projection avatar floating around that shows to position and orientation of the projector.

Source can be found in my repo at Google Code.

A long post, which I hope is useful for people. If not, let me know and I’ll keep it more concise next time!
And oh yes, happy New Year’s!

Finally, something new! But no, it’s not any of the bigger things I’ve been hinting at before. However, I’ve been hard at work at those (and other) projects for months, and ended up with tunnel vision and an annoying rut. I really needed to do something smaller again for some immediate visual fun. When Bartek showed me something on Twitter, I decided to take a step back from all the work, dig up some half-year old code and finish it asap. And so I did: that is a Fresnel Pixel Bender shader for Away3D, presented in the FresnelPBMaterial!

I’ve explained it before, but I’ll give a more general explanation this time around. Fresnel shading mimics the way light reflects off of (often translucent) surfaces such as water or glass. Looking head-on, you can see its proper surface (or see through it with refraction). However, if seen from a shallow angle, it reflects light (just imagine looking over a large body of water, or the mirage effect on the road on hot summer days).

Demos

I’ve thrown together some rather random demos to illustrate the point. There’s source for both. (A word of warning: apparently, on some FP10.1 betas, Pixel Bender causes it to throw an invalid input exception, which seems to be bogus.)

• Glass and marble head: Yes, the head appears to be floating in the middle of a cathedral of some blurry sort. Click to cycle through different property settings: glass, alien marble (I’m sure that’s a thing), and plain marble.
• Water surface: Using an animated normal map to illustrate how water could be rendered while using the fake refraction. It also shows the WaterMap util.

Another word of warning: slight implementation details might change in the next few days/weeks, but it should be nothing too dramatic!

I’ll probably be doing something more with this when I have the time, but nothing that would likely make it to the repository

It’s no secret that I like graphics. It’s the main reason why I play video games and it’s the main reason I got into programming. So obviously I was delighted to be invited and to join to the Away3D team last month. Inspired by my earlier Stok3d project (now on the backburner for a bit), I set off to create similar normal mapped pixel shaders, this time in full 3D. After some rough first patches (Stok3d was pretty simple since it only used DisplayObjects, flat planes), things have luckily shaped up, leading up to the first release!

The current state is a dot-release (3.4.2), so the exact implementation might still change while we’re working towards a shiny new 3.5.

The new shaders

So what is the difference with the previous shaders? Using Pixel Bender, the shading is calculated for every pixel in the texture, resulting in much more detailed and realistic lighting or reflections. Each shader requires an object-space normal map, which you can use to add detailed shading information without increasing the polycount of the mesh.

The shaders come in three flavors: environment map shaders, and single- and multi-pass shaders. Single-pass shaders take one PointLight3D and any AmbientLight3D on the scene to calculate the lighting, whereas multi-pass takes any number of lights of any type (AmbientLight3D, DirectionalLight3D, and AmbientLight3D). Important to note, tho, is that every light adds another pass and will be slower. Of course, if you’re only using 1 light, always use single pass.

Check out the following classes in Google Code:

• SphericEnvMapPBMaterial:  spheric environment mapping
• Cubic EnvMapPBMaterial: cubic environment mapping

• DiffusePBMaterial: Single-pass, adds diffuse lighting to the texture
• PhongPBMaterial: Single-pass, adds diffuse lighting and specular highlights, with support for specular maps
• SpecularPBMaterial: Single-pass, adds only highlights – can be used in combination with Prefab3D‘s prebaked lighting to create view-dependent specular highlights

• DiffuseMultiPassMaterial: Diffuse shading with multiple light sources
• PhongMultiPassMaterial: Phong shading with multiple light sources

Demos

Enough explanations, time for some demos! Right-click to view source:

• Cubic environment mapping
• Single pass shading: showing how a detailed normal map and specular map can enrich the shading
• Multi pass shading: using 3 lights (directional moonlight, firefly, and a flickering flame) and ambient

That’s it, enjoy! Feel free to drop by the mailing list for questions or read the official announcement! For now, I need to get some sleep before Flash on the Beach kicks off

Last week I posted an example of Environment Mapping using FP10′s native 3D and Pixel Bender. The reactions were quite positive, which motivated me to push the concept a bit further and create more shaders using Pixel Bender. These new additions all work in the same fashion, ie. as Filters which need to be updated whenever the target object (or if provided: light position) changes.

I created a project on Google Code for this, dubbed Stok3d (as I was positively stoked at that specific point in time). It’s a mini-library at this point, but in the future there’s potentially more to it than shaders; Z-sorting for instance: although it has been done already, it wouldn’t be a bad idea to create something specifically for Stok3d and have more functionality in one place. But… zat’s for ze futuah! At least for now, it will be easier to commit bugfixes and updates.

Demos

In the order from low cpu usage (and less visually interesting) to high cpu usage (and more interesting):

• Diffuse Shading: Basic shadowing, no specular highlights or normal map support
• *Phong Shading: Phong shading with support for normal and specular maps
• *Environment Mapping with Phong Shading: Combination of cubic environment mapping and Phong Shading, supporting normal and specular maps.

* Although Stok3d is distributed under the GNU GPLv3 license, the textures are NOT covered by this license. In particular, the blast door and hangar door textures, normal maps, and specular maps are made by Florian Zender (http://www.florianzender.com) and are used with his kind permission. Check out his work, it’s quite impressive!

Source

The source for the examples as well as the library can be found on Google Code. It’s available over svn or as a downloadable archive.