Ouch! My last post has almost been 5 months ago! Realizing this, I had two options: to feign my own death until the next update, or to give some sign of (what for me passes as) life. Let’s go with the second option and see if there’s anything useful I can pass on anyway.

Not having posted in such a long time doesn’t mean I’ve been slacking off, though. Quite the contrary; the past months have seen some of the most intense coding sessions I’ve went through in a long time. First a 3-month non-stop coding spree for Away3D, followed by continued development on Farbe. Hopefully, the result of both will be available in the (very?) near future, along with some other updates!

Now, instead of giving you demos or code, here’s some other things to help you develop.

(and no, I won’t be sharing any thoughts on the future of Flash)

Books!

Books form such an important part of my development process and knowledge, I’m a bit surprised myself that I haven’t really shared any titles here. Trying to ammend the situation, here’s some of the more influential ones I’ve read since I’ve started this blog (leaving out more general ones concerning design patterns etc):

• The Pragmatic Programmer, by Andrew Hunt and David Thomas: it’s a classic, and you should already have read it. So why am I still listing it? Because it often seems many Flash platform programmers stick to ActionScript/Flex-related books. Here’s a book that changes the way you think about your code, be it in form of design principles, tools or workflow. It’s a rather short read, so no excuses!
• Elements of Programming,  by Alexander Stepanov and Paul McJones: I found this one on a random trip to the bookstore. Approaching algorithms and data structures in a very formal and mathematical way, it forces you to – again - think differently about your code, but on a lower level (mainly concerning algorithms using data structures). What I love about this type of books is that you suddenly realize how differently you start tackling certain algorithmic problems. Perhaps not a book for everyone, but an interesting read nonetheless.
• 3D Game Engine Design, by David H. Eberly: (Thanks to Ralph for suggesting it way back ). “The Morgan Kaufmann Series in Interactive 3D Technology” has quite a few good books, but to me, this is the most valuable. Of all books I had at the time, this one taught me most about many 3D engine concepts. During the development of Wick3d, I couldn’t do without it.

IDE – IntelliJ

I’ve recently started using IntelliJ IDEA 9 for ActionScript and Flex development, after hearing Joa and the Flash Bum raving about it. Coming from Eclipse (Flex Builder and FDT), it took a while to get used to, but now I have I can’t imagine returning to Eclipse at all. I have been raving about it in real life, much to the annoyance of my co-workers, so it would be wrong of me not to mention it here.  The refactoring, template and code generation support is simply unmatched, there’s a very useful and clean UML diagram view, properly integrated Maven support, … AND I get to annoy the Java/back-end team directly inside the IDE

To think I was once had to code C in MS-Dos Edit, how times have changed…

Edit: The Flashbum just wrote an article on InsideRIA about how to get started. Read all about it!

That’s about it. I hope this sign of life is useful to anyone after all! If not, I promise there will be more real updates soon!

It’s no secret that I like graphics. It’s the main reason why I play video games and it’s the main reason I got into programming. So obviously I was delighted to be invited and to join to the Away3D team last month. Inspired by my earlier Stok3d project (now on the backburner for a bit), I set off to create similar normal mapped pixel shaders, this time in full 3D. After some rough first patches (Stok3d was pretty simple since it only used DisplayObjects, flat planes), things have luckily shaped up, leading up to the first release!

The current state is a dot-release (3.4.2), so the exact implementation might still change while we’re working towards a shiny new 3.5.

The new shaders

So what is the difference with the previous shaders? Using Pixel Bender, the shading is calculated for every pixel in the texture, resulting in much more detailed and realistic lighting or reflections. Each shader requires an object-space normal map, which you can use to add detailed shading information without increasing the polycount of the mesh.

The shaders come in three flavors: environment map shaders, and single- and multi-pass shaders. Single-pass shaders take one PointLight3D and any AmbientLight3D on the scene to calculate the lighting, whereas multi-pass takes any number of lights of any type (AmbientLight3D, DirectionalLight3D, and AmbientLight3D). Important to note, tho, is that every light adds another pass and will be slower. Of course, if you’re only using 1 light, always use single pass.

Check out the following classes in Google Code:

• SphericEnvMapPBMaterial:  spheric environment mapping
• Cubic EnvMapPBMaterial: cubic environment mapping

• DiffusePBMaterial: Single-pass, adds diffuse lighting to the texture
• PhongPBMaterial: Single-pass, adds diffuse lighting and specular highlights, with support for specular maps
• SpecularPBMaterial: Single-pass, adds only highlights – can be used in combination with Prefab3D’s prebaked lighting to create view-dependent specular highlights

• DiffuseMultiPassMaterial: Diffuse shading with multiple light sources
• PhongMultiPassMaterial: Phong shading with multiple light sources

Demos

Enough explanations, time for some demos! Right-click to view source:

• Cubic environment mapping
• Single pass shading: showing how a detailed normal map and specular map can enrich the shading
• Multi pass shading: using 3 lights (directional moonlight, firefly, and a flickering flame) and ambient

That’s it, enjoy! Feel free to drop by the mailing list for questions or read the official announcement! For now, I need to get some sleep before Flash on the Beach kicks off

After a futile exploration of sparse voxel octree ray casting using Alchemy (which was fun but hopeless), I turned towards another technique for volume rendering, using view-aligned slices. The approach is not much different from the rendering of this older experiment, in which the slices were aligned to the object itself. Again, we’re using the same technique to create and read from the 3D texture (which is static in this case): ie. a set of cross sections placed next to eachother. CT scans are wonderful for this:

Not that the image above is just a crop-out, we need a lot more to make it look decent (I used 32 cross sections).

Rendering the slices

When using view-aligned slices, they typically won’t be aligned to the texture’s slices, as illustrated in the image to the right (yes, my graphic skills are EPIC!). The point p is any point on any view-aligned slice. We need to know where it is in the texture’s 3D space. This is simply a change of basis transformation, where both bases are defined to have the same origin. In our specific case, eye space is world space, so all we have to do is multiply p with the inverse of the object’s delta transformation matrix. Since the result will usually lie between 2 slices of the 3D texture (as in the illustration), we sample both texture slices with constant x and y coordinates and interpolate the colour values. This approach is not 100% correct, since the interpolation should also be aligned to the view. However, for this purpose, it’s a good trade-off for some extra performance.

As this needs to be done for every pixel on every slice, we’re doing these calculations through Pixel Bender. And that’s how it works in general lines. There’s some translations and scaling going on as well to ensure a uniform and properly centered transformation. If you’re still interested, you can check the source for that. Important to note is that half the slices in the back are actually culled for a worthwhile performance boost. They don’t really contribute all that much to the final image after all.

Demos

Click and drag to rotate the pitbull skull in all demos:

• High quality: using a 4096×128 (ie 128×128x32) texture with 16 visible slices
• Low quality: using a 2048×64 texture with 16 visible slices
• Ultra quality: using a 4096×128 with 50 visible slices (slow!)
• Ultra high quality static rendering with low quality dynamic rendering: getting the best of two worlds
• Source

Today’s update on Stok3d is perhaps not as useful as the previous post, but I certainly had fun working on it. Or as we say in Dutch with a word blatantly stolen from German: it’s “spielerei”.

Demos:

I’m going to post the demos first this time. Saves you some scrolling effort, because the explanations below are rather boring

• Parallax Mapping : Move and turn towards the edges of the screen to see the extrusion of the texture best. A PhongFilter is added as a second filter, making it slower but the effect becomes more obvious.
• Water Shading 1 – Ocean : Reflects or refracts light depending on the view angle and the surface relief. It animates a perlin noise filter to generate a water heightmap.
• Water Shading 2 – Ripples : Same thing, but with a simple drawn ripple effect. The difference between refracted and reflected light is more obvious here.

Edit: Even if you have Flash Player 10, you still might get an update request. That is because these demos require the version of 10.0.22 concerning recent Pixel Bender bugfixes.

The source code for these demos can also be found on Google Code.

Useful updates

Some updates I did involved some bugfixes and performance-related updates. I also added a NormalMapUtil class, which provides a basic API to generate and manipulate normal maps. The main features are the generateFromHeightmap and drawFromHeightmap methods. Since height maps (or bump maps) are generally easier to come by (and to make), these methods generate a normal map for you. generateFromHeightMap creates a new BitmapData, drawFromHeightMap uses an existing BitmapData that you provide (useful if you need to generate one on every frame). The NormalMapUtil class furthermore allows you to invert the components of the normals, in case a normal map reflects the light in the wrong directions.

Water Shading

A new shader filter that was added is the WaterShadingFilter. If you remember your high school physics, depending on the view angle, the surface either reflects the light or lets it pass through and refracts it (which is called the fresnel effect).  To put it simply, when looking at water at a shallow angle, it seems like a very reflective surface, but when looking straight down into it, you can see through it but it’s a distorted view. The reflection uses a combination of environment mapping and phong shading, while the refraction is a simple displacement mapping technique. The DisplayObject to which the filter is applied is used as what’s underneath the surface, ie: the refracted light. The ripples seen in the demo are not made by the filter, but are custom written to manipulate the normal map.

Parallax Mapping

Another new filter is the ParallaxFilter, which performs (you guessed it) parallax mapping. It’s a technique like bump and normal mapping, in the sense that it tries to give more depth to a 2D texture. It does so by displacing texture coordinates based on a height map and view direction to the coordinate that it would normally have in 3D space. This causes the texture to look extruded and more detailed. For more (and better) information, check the article on wikipedia. Stok3d implements an iterative variation. It’s a bit slower but takes care of overlap issues and can handle sharp edges.

That’s it! For now, it’s back to Farbe, and… some other things

Last week I posted an example of Environment Mapping using FP10’s native 3D and Pixel Bender. The reactions were quite positive, which motivated me to push the concept a bit further and create more shaders using Pixel Bender. These new additions all work in the same fashion, ie. as Filters which need to be updated whenever the target object (or if provided: light position) changes.

I created a project on Google Code for this, dubbed Stok3d (as I was positively stoked at that specific point in time). It’s a mini-library at this point, but in the future there’s potentially more to it than shaders; Z-sorting for instance: although it has been done already, it wouldn’t be a bad idea to create something specifically for Stok3d and have more functionality in one place. But… zat’s for ze futuah! At least for now, it will be easier to commit bugfixes and updates.

Demos

In the order from low cpu usage (and less visually interesting) to high cpu usage (and more interesting):

• Diffuse Shading: Basic shadowing, no specular highlights or normal map support
• *Phong Shading: Phong shading with support for normal and specular maps
• *Environment Mapping with Phong Shading: Combination of cubic environment mapping and Phong Shading, supporting normal and specular maps.

* Although Stok3d is distributed under the GNU GPLv3 license, the textures are NOT covered by this license. In particular, the blast door and hangar door textures, normal maps, and specular maps are made by Florian Zender (http://www.florianzender.com) and are used with his kind permission. Check out his work, it’s quite impressive!

Source

The source for the examples as well as the library can be found on Google Code. It’s available over svn or as a downloadable archive.

Since Flash Player 10 introduced native 3D functionality, the world of “postcard 3D” rejoiced as doing simple 3D tasks became much easier… Just as long as it did not involve z-sorting or shading effects.

The z-sorting has been tackled before, and here’s an attempt at implementing the second idea. Partly out of boredom and partly because I needed it as a first step towards another experiment, I created a simple cubic environment map effect, just in case you’d want your surfaces to be reflective. The filter itself is using Pixel Bender – what a surprise!

There’s some other possibilities I’m thinking of (phong shading for example), but it’d be good to know if there actually is any interest in stuff like this. I don’t want to be wasting my time either

Cubic Environment Maps

Environment mapping is essentially a fake but relatively fast way to give the illusion of surface reflection. There are two common approaches: spherical and cubic mapping. The first maps the environment texture surrounding the scene on a sphere before mapping it onto the surface, the latter does it using a cube. Spherical mapping is usually faster and easier to use (it only requires 1 or 2 textures instead of one for every face of the cube), but sadly it doesn’t look very natural on a flat surface. Cubic mapping, on the other hand, looks better and is more commonly used these days. For more information, check this article on wikipedia.

So what do you need? Basically, you need to find a cube map and divide it up into 6 seperate images as illustrated below. You can assign these images to the EnvMapFilter class constructor.

When changing the surface position, scale or rotation, be sure to call the update method and reassign the filter to the target DisplayObject. All this is shown in the source.

Normal maps

It’s not a requirement, but if you want, you can assign a normal map to the filter. For every pixel, normal maps indicate the slope of the surface at any point and cause the reflections to be distorted. This gives the surface the appearance of having relief. A quick search on google or some texture libraries should yield plenty of them

Blabla yeah whatever!

Okay okay, enough chatter.

• Demo – just follow the instructions (FP10.0.22 required, so you might get an upgrade request)
• Source

Enjoy!

One final thing I wanted to achieve with smoke effects is to port the 2D simulation to 3D. Early results in pure AS3 were pretty disappointing, so again I turned to Pixel Bender and its speedy goodness. Still, the main obstacle was of course performance. The first step was to optimize the original 2D version by doing more in a single filter, throwing out some repetitive calculations performed per pixel, etc. And obviously, adding an extra dimension means more calculations. This system is grid-based, so the amount of nodes increases a lot! As a result, the grid size is reduced to 18×32x12 , which is leaning dangerously close to the realm of visual crud. Still, as an experiment, I consider it acceptable, and it’s using some tricks that might be worth sharing. Perhaps someone can do something better with this

I’ll explain a bit on how it works since I usually forget that part (if you’re not interested, just scroll down a bit). The actual fluid solver is based on the work of Jos Stam, and a lot has been written on the subject already.

3D textures

Pixel Shaders for 3D graphics such as GLSL have support for 3D textures. Since Pixel Bender is made for 2D, we have to find a different way to map a 3D grid to work with Pixel Bender. The solution is so straightforward that it’s hardly worth mentioning, but since I haven’t seen it done before, I will explain it anyway. All slices of the texture/grid along the z-axis are simply placed next to eachother, as pictured below:

Analogous to the representation of a 2D texture in a 1D array, the 2D coordinates are simply found by coord2D(x,y,z) = (x+gridWidth*z, y), or in a 1D array as coord1D(x, y, z) = (x+y*gridWidth*gridDepth+z*gridWidth). Told you it’s pretty pointless

Rendering

My first thought was to consider the grid as a set of voxels and use a raymarching algorithm to render it, although I knew that would end up being way too slow. Luckily, I found a much simpler solution that worked well enough. Depending on the world axis that is closest to the view vector, the grid is sliced up in bitmaps aligned to the grid’s local axes. Looking at the “box” head on, it’s simply divided in planes (parallel to the local XY plane) from back to front, when looking more from the left, it’s divided from left to right (parallel to the local YZ plane) and so forth. Using these slices, they’re simply placed on the stage as Bitmap objects using FP10’s 3D functionality. Luckily, the lack of depth sorting does not seem like such an important issue to create the illusion of a volume filled with smoke.

Finally

I’ll close up with some good news: I think I’m done with smoke for now! It’s not a promise, though… Moving on:

• The demo – The box is invisible at first, in the middle of the screen. Fill it up with smoke and it’ll become visible.
• The source – To get performance up a bit, there’s some guerilla-style coding. Enter at own risk!

Thanks to Joa for running his AS3V-tool on it – I can’t wait for it to go public! I don’t think I caught all violations, tho

If you’ve played any video games the past few years, you’ve definately seen the HDR (High Dynamic Range) effect. Creating a more brilliant colour spectrum, it’s both popular in photography and 3D rendering. In games it typically enhances the brighter areas and makes light spill over into darker areas (the so called light blooming effect). Using some settings, it can also create a more blurry atmospheric surrounding. Being a big game fan (or rather gfx-fetishist), it’s an effect I’ve always wanted to do. Using a tiny Pixel Bender kernel and some Flash filters, I came up with this.

A demo showing the effect and its parameters on a simple image can be seen here: Demo | Source .

But let’s face it, when doing HDR, we want it in 3D  Rob Bateman kindly gave me permission to use his shader demo for Away3D (a 3D engine I’m growing more and more fond of). This demo in particular suited the use of HDR perfectly, so many thanks! I just added the HDRContainer class to the stage, setting the View3D as the target, and voila!

See HDR in Away3D (compare with the original to see the difference)

No source for that one tho, as 1) it’s not my demo (apart from 2 extra lines of code), and 2) it’s pretty much the same as the first demo

It should be similar to use the effect on any DisplayObject, for that matter: pass it through to the HDRContainer’s constructor and add the HDRContainer to the stage.

This blog’s been a bit silent the past couple of weeks, but I haven’t been idling at all – quite the opposite. Even with fresh copies of Dead Space, Fable 2 and Fallout 3 laying too near for comfort, most of my free time has been taken up by Wick3d (1 | 2). Recent developments in Papervision3D and Away3D (the choices for commercial-grade Flash 3D) only made me more excited to do some extra studying and experimentation. Just to clear things out now – there won’t be any demos in this post, but I will mention what I’ve been working on, and hope to merge with the Wick3d repository sooner than later

• Lighting/Shaders: The old Wick3d (before I started over, that is) had some basic support already, but shading is going to make an improved reentrance.
• Correct Z-Sorting: a tricky one. Since a stencil (z-)buffer isn’t an option in Flash, the only way I know how to sort triangles correctly is the Newell-Newell-Sancha algorithm (an extension to the simple painter’s algorithm). However, since it’s an O(n²) algorithm it can slow down to a crawl rapidly with enough triangles. Organizing the triangles in a quadtree (similar to PV3D’s latest addition) could improve performance, but still – per quad – there’s the n² issue. I’m sure there’s better ways to solve this problem and I’ll keep up the research. Any tips on this subject would be most welcome, of course
• Working with the BSPRenderPipeline has been reworked and now supports dynamic objects moving about in the BSP tree (this one’s actually done, but not comitted to the trunk yet - check out the dev branch if you’re feeling brave  ). Create a BSPWorld3D instead of a World3D, passing the static base to the constructor, and add dynamic children with addChildren as usual. A demo with source will follow soon, with some luck.
• Redone Collada support. It’s still very basic, no hierarchy, but parsing is much faster. It will be improved whenever needed. Parsing files is not my preferred pass-time, so I tend to keep it at this point until I personally need more support.
• Restructuring of certain parts of the engine. For example, Rasterizers are now classes on their own, so different types can be used in any render pipeline.
• Various optimizations and trivial updates.
• Thinking about FP10 and how I could harness the power of drawTriangles when dealing with (virtually) a polygon soup. I guess you’d still be drawing every triangle separately without sharing the vertices.

Heh, at least I’m not slacking off! It might take a while before these updates are comitted to the main trunk, so I thought I’d give this rundown of things to come – hopefully! Any brilliant insights? Let me know