Since I’ve been working on Away3D 4.0 (Broomstick) for the past few months, it was pretty hard to find the energy to keep this blog up to date, and the promised part 2 of Hacking the Lite kept getting delayed. And delayed. And then delayed some more. Having access to the GPU also caused some debate on whether this series would still be relevant enough. However, after looking at my initial goals for it, I decided it is. After all, the idea was to inspire people to start playing around, and to get people to think in terms of using shaders. Most of the concepts outlined here can be used in Molehill as well. And of course, it’s still some time before we can use the GPU in actual commercial projects So on with the show! This part of the series will build heavily upon the previous part, so it’s perhaps a good idea to review that again.

Shadow maps

Casting shadows is often done using shadow maps. In this case, we’d render a “depth map” of the scene from the light’s point of view. This way, we have a texture containing the distances to the objects closest to the light for every projection ray (which is a single pixel in the shadow map). When rendering the scene, we project every point we are rendering to that shadow map. We can then compare the depth value of the point-to-be-rendered with the value in the shadow map. If it is further away from the light, it must be in shadow and should not be lit. This approach works well, but for current versions of Flash, it’s not very feasible. But, since we’re hacking, there’s other options!

Augmenting projective texture mapping

If you take a look at the schematic on the left (click to enlarge), showing where a shadow should fall, you can see it’s much like projection mapping. The sphere is being perspectively projected unto the floor plane with the light as the projection’s origin. This is the exact same projection that projects the light texture onto the objects. As you can see, if we project the sphere onto the light’s texture first, and project that in turn to the floor, the shadow region remains the same. As a result, we could actually simply project and draw the sphere into the light map before it is applied to the floor.

Order and form

To render a scene with multiple objects, we have to take care to render things in correct order. We need to draw the objects closest to the light first, because they will obviously be casting shadows on object further from the light. Starting out with a clean light map (ie.containing no shadows), we project it on the closest object and then draw that object’s shadow into the light map. Move on to the second closest object, light it, and draw the shadow again. And so on until the last object was reached.

That works great for convex objects, as long as they don’t intersect. For concave or intersecting convex objects, it’s possible that object A casts shadows on object B as well as the other way around. Since we’re rendering everything in a strict order, that’s simply not a possibility. However, this error should generally be acceptable. Another problem is self-shadowing for concave objects, which is impossible using this “cheat”.

Demo and source

Fake shadow mapping demo: really just an adapted version of the previous one.

The source is again on Google Code!

Exciting times afoot! In the not too distance future, we’ll finally get what we’ve been nagging about! I’m of course talking about Molehill, the new hardware-accelerated API Adobe is working on. If you’re curious what we’re doing with Away3D, take a peak at the demo we did with the guys from Evoflash for Adobe MAX. Old news, I know. Moving on.

All that and more is still some time in the future, and in the meanwhile we need to make do with what we currently have available. As part of a session at AUG XL and FlashPlatform, I gave some examples of what we currently can do with Away3D Lite if we’re willing to get our hands a bit dirty. As examples I explained some effects I did for the Spiral Out demo. Now, it’s time to clean up the code mess and explain a thing or two to a slightly broader audience

Basically, what I’ll do is show some things that involve making assumptions that prevent the code from being scalable (ahhh… you got me… that’s just a nice way of saying I’m cheating!). The first step isn’t actually cheating at all, and it’s an old and pretty standard shading technique. However, we’ll need this for part 2 in which we’ll do nothing but cheating, and since no one ever seems to do shaders in Lite I thought it’d be a good way to get started.

This it what we’ll end up with. Not the most attractive demo, but it’ll do. Lengthy explanations ahead, for which the source can come in handy (under examples/projectionMapping)

Projective Texture Mapping

Projective texture mapping, as the name clearly suggests, is a texture mapping technique that mimics a light projecting a texture on objects in the scene (think of a projector slide, a lamp shade’s projections on the walls, …). As I said, there’s nothing new or special about this; it’s been done countless times before. The picture on the right pretty much shows all there’s to it (click to enlarge).

Essentially, we place the “light texture” in front of the light. All we need to do, is draw a line from the position we’re currently shading (the “target point”) to the light (the “source point”) and figure out where that line intersects the plane. Grab the light texture’s colour at that position and multiply it with the object’s base texture colour to get the final shaded colour. We can also incorporate additional shading calculations, particularly normal mapping.

Exactly how we calculate the intersection is up to you. You can do a line-plane intersection calculation, or you can – surprise surprise – use a perspective projection matrix to project the target point onto the texture. So rather than projecting the texture onto the object, we’re effectively projecting the object on the texture first. The latter approach is much more flexible and intuitive, since it’s exactly how a camera projection works (in part 2, we’ll see that this will come in handy too).

Shading in Away3DLite

Away3DLite, minimal as it is, has only very limited support for shading. There’s a Dot3BitmapMaterial that no one seems to know about, though, courtesy of Mr. Bateman I recommend taking a look at that code, just to see how you could implement a custom shader. At a minimum, all you’d need to do is extend BitmapMaterial and override the updateMaterial method (don’t forget to include and use the namespace arcane). In there, you can do anything you like as long as you end up passing a valid BitmapData object to the _graphicsBitmapFill.bitmapData property. For shading purposes, it’s best to keep a second BitmapData object to which we’ll be rendering the shaded version of the texture, leaving the original _bitmap object intact for subsequent frames. Of course, for any advanced shading on BitmapData objects, Pixel Bender is king, so we can run a ShaderJob in there.

Taking a look at the source for the ProjectiveTextureMaterial, that’s pretty much exactly what is happening, with one difference. We’re exposing a manual update method rather than overriding updateMaterial. This seems rather odd (and in this particular case it is) but we’ll need that next time (remember, we’re hacking anyway)

There’s of course a final caveat, in that we don’t have any position information since we just have a bunch of BitmapData objects, no actual geometry. And that is of course something essential if we want to do any projecting at all. For that we can take the same approach as the Away3D Pixel Bender shaders do: position maps. Whereas a normal map encodes the normal for every texel, a position map encodes the object position at that point. In Away3D, the pixel shaders generate a position map automatically from the geometry when a material is created, but in Lite, we’ll settle for creating them manually. This actually makes it possible to add details from a height map, providing the shader with much more detailed topological information than what would have been generated from a bunch of triangles. As a result, the projected map can be subtly displaced as it would on a real uneven surface. It’s not very hard to create position maps procedurally (which is part 3) if you keep in mind that xyz maps to rgb and 0×00-0xff for each colour channel maps to the minimum and maximum bounds of the respective coordinate (so where x == minX, r would be 0).

Demo and source

To see the effect in action, check out the demo. Move the mouse to move the projector, click and drag to move the camera, and use the mouse wheel to move the projector closer or farther from the scene. There’s also a small projection avatar floating around that shows to position and orientation of the projector.

Source can be found in my repo at Google Code.

A long post, which I hope is useful for people. If not, let me know and I’ll keep it more concise next time!
And oh yes, happy New Year’s!

As a continuation of an earlier post, here’s a variation of the Fresnel Pixel Bender material. Still obviously inspired by Bartek‘s Unity3D shader, it became a glass material with support for chromatic aberration (what causes the rainbow effect in refracted light). The result could come in handy in some situations, so I thought I’d share and commit it to the Away3D trunk as the GlassMaterial.

The material uses the same environment map to do (fake) refractions and reflections, so it’s not actually refracting the real scene around it. However, if you make a sphere map rendering of your scene, the effect can be convincing enough.

The demo shows a couple of default primitives and several settings of the material (such as glass colour and reflection strength), and also uses a new primitive called TorusKnot (which is a (p,q) torus knot). Consider that a tribute to a demo shown by Ralph during his presentation at FITC Amsterdam . Click to cycle through the different settings, and check the source to see how it all works.

Update: As Makc pointed out in the comments (and rightly so), the refraction looked rather icky, so I updated it that it uses linear sampling. It’s a bit slower but looks much better.

Today, we’re happy to announce the release of Away3D 2.5.0 and 3.5.0. Along with other features and changes, of which you can read more in the official announcement, it contains the new BSP/PVS support! I’ve slaved months on this, so I’m glad it’s finally made public! Keep in mind that we’re always improving it, so it’ll only get better from here So what’re all these abbreviations about?

(Warning: a long-winded post ahead. If you’re not interested in the details, stick to the official announcement!)

Binary Space Partitioning

BSP is a scene graph that divides a static scene up by splitting it recursively along planes into two half-spaces. If after splitting, the geometry in that subspace is convex, no further subdivisions are needed. In the end, we have a binary tree containing convex meshes at the leaves. What’s the use, you may ask? Well, since all the parts are convex, we don’t need to do any per-triangle sorting. By correctly traversing the tree and rendering the parts in the right order, the sorting is faster AND 100% correct. Specifically for Flash 3D, that means we don’t need to subdivide triangles to try and resolve z-fighting. Furthermore, it also greatly speeds up frustum culling and collision detection. This only works for static scenes however. There are dynamic bsp trees, but they don’t support the biggest speed-up, being the PVS.

Binary Space Partitioning on Wikipedia

The Potentially Visible Set

When we have a static BSP, we can use this data to determine which leaf nodes are visible from any given leaf node. This dataset is called the PVS (Potentially Visible Set). It allows us to prevent most of the occluded geometry from being rendered (which, usually, is most of the total geometry). Needless to say, this can lead to huge speed improvements. It’s the same technique games such as Quake, Half-Life, Unreal, … used. However, this speed improvement comes at a price (or what did you think? ), which is explained in the “Building” section of this post.

This little bastard was a pain to do. It’s the reason days turned to months of my evenings to get right, and I definitely couldn’t have done it without the help from this thesis: View Space Linking, Solid Node Compression and Binary Space Partitioning for Visibility Determination in 3D Walk-Throughs by Joel Anderson. Also, props to Joa for giving me some optimization tricks and food for thought!

Potentially Visible Set on Wikipedia

Building

Fabrice did a great job implementing the BSP/PVS builder into Prefab3D, so you can bake your lighting and generate the BSP/PVS as AWData in one programme. When compiling a model with PVS, there’s some guidelines you should keep in mind. No, let’s call them rules instead. Not obeying these could result in the build getting stuck, poor collision detection, or some random unpredictable results.

• Your model must be completely “sealed”. No leaks into “nothingness”! This is important because the PVS would try to find visible data from that leak and get stuck.
• Work on a grid. Make sure your vertices are snapped to a grid which is not too coarse. If not, the BSP splitting could create triangles that are too small, causing havoc in the form of floating point errors, too small triangles, inefficient splits, etc…
• Try to occlude a lot. Don’t make too much geometry visible from any part of the model. This speeds up the model during rendering as well as the PVS build.
• Axis-aligned walls/ceilings/floors work best. They’re fast to do calculations on and if you model somewhat smartly, they form a better “space splitter”.
• Sometimes it’s good to keep small details out of the BSP geometry, and add them using the regular addChild method as this keeps the trees cleaner and more efficient.

Finally, there’s a LOT of settings, but you should probably not have to mess too much with them unless you really know what you’re doing. Soon, we’ll have some proper tutorials explaining the nitty-gritty of building (and we might have some other tools to make things easier on you).

Check the video showing a build in action!

Demos

• The Hacienda Experiment (Beta): Fabrice and I created this for Adobe’s keynote at FITC Amsterdam in a very limited amount of time, before the BSP functionality was even finished. Lots of stress and insecurities, building the AWData exporter, etc. So, forgive us if there’s still some bugs in this version (If you get stuck, jumping out using the space bar usually does the trick )
• Bunker Demo: Made by Fabrice to illustrate how easy it is to export BSP/PVS data from Prefab3D (see video mentioned before). It also shows how fast it can run with good occlusion. (Check the source)

Thanks for sticking around for such a long post, but hey, with the amount of work gone into this, I reserve the right to bore you to death with it!

It’s no secret that I like graphics. It’s the main reason why I play video games and it’s the main reason I got into programming. So obviously I was delighted to be invited and to join to the Away3D team last month. Inspired by my earlier Stok3d project (now on the backburner for a bit), I set off to create similar normal mapped pixel shaders, this time in full 3D. After some rough first patches (Stok3d was pretty simple since it only used DisplayObjects, flat planes), things have luckily shaped up, leading up to the first release!

The current state is a dot-release (3.4.2), so the exact implementation might still change while we’re working towards a shiny new 3.5.

The new shaders

So what is the difference with the previous shaders? Using Pixel Bender, the shading is calculated for every pixel in the texture, resulting in much more detailed and realistic lighting or reflections. Each shader requires an object-space normal map, which you can use to add detailed shading information without increasing the polycount of the mesh.

The shaders come in three flavors: environment map shaders, and single- and multi-pass shaders. Single-pass shaders take one PointLight3D and any AmbientLight3D on the scene to calculate the lighting, whereas multi-pass takes any number of lights of any type (AmbientLight3D, DirectionalLight3D, and AmbientLight3D). Important to note, tho, is that every light adds another pass and will be slower. Of course, if you’re only using 1 light, always use single pass.

Check out the following classes in Google Code:

• SphericEnvMapPBMaterial:  spheric environment mapping
• Cubic EnvMapPBMaterial: cubic environment mapping

• DiffusePBMaterial: Single-pass, adds diffuse lighting to the texture
• PhongPBMaterial: Single-pass, adds diffuse lighting and specular highlights, with support for specular maps
• SpecularPBMaterial: Single-pass, adds only highlights – can be used in combination with Prefab3D‘s prebaked lighting to create view-dependent specular highlights

• DiffuseMultiPassMaterial: Diffuse shading with multiple light sources
• PhongMultiPassMaterial: Phong shading with multiple light sources

Demos

Enough explanations, time for some demos! Right-click to view source:

• Cubic environment mapping
• Single pass shading: showing how a detailed normal map and specular map can enrich the shading
• Multi pass shading: using 3 lights (directional moonlight, firefly, and a flickering flame) and ambient

That’s it, enjoy! Feel free to drop by the mailing list for questions or read the official announcement! For now, I need to get some sleep before Flash on the Beach kicks off

Today’s update on Stok3d is perhaps not as useful as the previous post, but I certainly had fun working on it. Or as we say in Dutch with a word blatantly stolen from German: it’s “spielerei”.

Demos:

I’m going to post the demos first this time. Saves you some scrolling effort, because the explanations below are rather boring

• Parallax Mapping : Move and turn towards the edges of the screen to see the extrusion of the texture best. A PhongFilter is added as a second filter, making it slower but the effect becomes more obvious.
• Water Shading 1 – Ocean : Reflects or refracts light depending on the view angle and the surface relief. It animates a perlin noise filter to generate a water heightmap.
• Water Shading 2 – Ripples : Same thing, but with a simple drawn ripple effect. The difference between refracted and reflected light is more obvious here.

Edit: Even if you have Flash Player 10, you still might get an update request. That is because these demos require the version of 10.0.22 concerning recent Pixel Bender bugfixes.

The source code for these demos can also be found on Google Code.

Useful updates

Some updates I did involved some bugfixes and performance-related updates. I also added a NormalMapUtil class, which provides a basic API to generate and manipulate normal maps. The main features are the generateFromHeightmap and drawFromHeightmap methods. Since height maps (or bump maps) are generally easier to come by (and to make), these methods generate a normal map for you. generateFromHeightMap creates a new BitmapData, drawFromHeightMap uses an existing BitmapData that you provide (useful if you need to generate one on every frame). The NormalMapUtil class furthermore allows you to invert the components of the normals, in case a normal map reflects the light in the wrong directions.

Water Shading

A new shader filter that was added is the WaterShadingFilter. If you remember your high school physics, depending on the view angle, the surface either reflects the light or lets it pass through and refracts it (which is called the fresnel effect).  To put it simply, when looking at water at a shallow angle, it seems like a very reflective surface, but when looking straight down into it, you can see through it but it’s a distorted view. The reflection uses a combination of environment mapping and phong shading, while the refraction is a simple displacement mapping technique. The DisplayObject to which the filter is applied is used as what’s underneath the surface, ie: the refracted light. The ripples seen in the demo are not made by the filter, but are custom written to manipulate the normal map.

Parallax Mapping

Another new filter is the ParallaxFilter, which performs (you guessed it) parallax mapping. It’s a technique like bump and normal mapping, in the sense that it tries to give more depth to a 2D texture. It does so by displacing texture coordinates based on a height map and view direction to the coordinate that it would normally have in 3D space. This causes the texture to look extruded and more detailed. For more (and better) information, check the article on wikipedia. Stok3d implements an iterative variation. It’s a bit slower but takes care of overlap issues and can handle sharp edges.

That’s it! For now, it’s back to Farbe, and… some other things

When looking at my blog stats, the most popular post by far is the old Ripple effect I did some time ago, which was based on an old demoscene trick. For a while, I’ve wanted to do a more physically correct version based on shallow water equations. Since the watercolour effect I did was based on a similar variation (as well as the image bleeding in the previous post), I decided to throw together a quick test to see if it could be turned into a new iteration of the old ripple effect. It’s mainly done for sake of experimentation, and if you really want a ripple effect I suggest using the prior one. Although the dynamics are more interesting, this one is slower, but I’m pretty happy with the performance considering the amount of calculations and the grid size (the demo has a grid of 200×200).

• Demo
• Source – There’s a few parameters to play around with, but take care: the simulation might just blow up with extreme settings

Some day I might try out a sim with the real shallow water equations, just in case you haven’t gotten violently sick from all the fluid simulations lately (I haven’t, as seems)

Thanks to Ralph Hauwert’s blogpost pushing a baffling >300.000 particles using Alchemy, I finally got the much needed motivation to try out Alchemy myself. To be able to get started asap, I decided to return to something I’ve done before: fluid and smoke dynamics, but with a different algorithm (see Mike West’s article on Gamedev). It took me a while to figure it out, but it’s up an running! It’s a faster algorithm, but less physically correct.

While I typically would do most of these kinds of calculations in Pixel Bender, for the examples below, I decided to try it all in C and see how far Alchemy can be pushed. The grid is bigger than before, while the framerate is steady around 20 on my machine. Ralph has suggested some improvements for the rendering step already (thx! ), so I’ll have some more experimenting to do

It was great to return to C, especially since it’s been 8 years since I’ve written my last line in that language. Quite a rush of nostalgia!

Source is included with the demos, tho it’s not the most reusable I’ve ever written

• Smoke simulation: add and move around smoke (Source)
• Milk and Ink (fluid within fluid): milk is healthy, until you add ink! (Source)

The HiRes FPS counter is done by Mr. Doob.

On a rare occasion, instead of posting code and experiments, I like to post some things I’ve done at my job. Especially when it illustrates a practical use of some of my earlier experiments. So here we go!

For a project for the University of Ghent at Nascom (via Saatchi & Saatchi), I had to create a front-end application to be displayed on a big 12m² billboard, placed in Ghent. Without going into too much detail of the whole concept (which you can read at the official blogpost), it consisted out of a set of mini-applications allowing people to text mobile messages and interact with the display (show messages, a poll, …). being all booked up, I had to sacrifice some free time to be able to work on it. The reason I did was the same reason I decided to use Flash Player 10: a perfect excuse to work on the earlier 2D cloth simulation and turn it into stage curtains on a whopper public screen. Blatant self-exposure? Nah :p

Below, you can see a video recording showing it in action. The curtains open up after someone sends a text to the service, showing a random picture or video. Apologies for the low quality, all we had was a photo camera, and led lights don’t like to be filmed

Thanks to Jan Nikolaas Gijsen for editing the video! And I promise, this will be the last cloth-related post – at least for some time

A project I’m currently involved in inspired me to completely rewrite my old Curtain class into something more stable and versatile. Using a character physics method based on Verlet integration, and adding some properties for friction and gravity, it resulted in a 2D cloth simulation (at least after some updates I just did since I needed to get away from work for a bit). The curtain itself is drawn using the new drawTriangles API for Flash Player 10.

Anyone interested in Verlet integration (or scripted animation in general) should check out Keith Peters‘ book AdvancED ActionScript 3.0 Animation.

On to the demo! Right-click to view source. Not commented due to lack of time, but it shouldn’t be too hard to figure out