With Flash on the Beach behind us and having had time to process everything, I can safely say it may have been the best edition yet. A great way to get inspired and meet old as well as new friends, and every year it becomes harder to leave. So thanks John for organizing yet another kick-ass festival (yeah it’s really more like a festival than a conference ).

One of the things I swear by every year, is that you haven’t experienced Flash on the Beach if you didn’t catch Joa Ebert‘s session, especially since he started to work on his Apparat suite. Needless to say, I was very honoured when he asked me to contribute some Pixel Bender shaders to show his new JITB player running on the GPU. For those not aware of JITB, it’s essentially a lean & mean Java-based Flash player, with all the niceties the Java runtime provides. Read more about it here.

Results

I ended up writing two ray tracers (or rather, ray marchers).

Yeah, yet another quaternion Julia set. I mentioned last time that they look much better raytraced, and I hope you’ll agree. The sound synching is as interesting as I could get it with only left/rightPeak being supported and a limited amount of time
Based on the paper “Ray Tracing Deterministic 3-D Fractals” by John C. Hart et al.
Music: Aphex Twin – Xtal

It’s a bit hard to tell, but the terrain marching also has dynamic lighting and shadow casting. The frame rate in the video is actually quite bad, due to the screen capture.
Based on Inigo Quilez’ Terrain Raymarching article

Source

In the spirit of sharing, happy bunnies and double rainbows OMG, I provided the source to these projects, so you can run it yourself.

Get it here!

Of course, you’ll need to run it using JITB. Check Joa’s post on how to get things running.
Also, the download is a bit heavy due to big-ass terrain images (just because, y’know, now I can )

DISCLAIMER: run it at your own risk. If your machine/GPU is known to overheat easily, it’s best not to try it at all (Case in point – sorry dude  ).

Keep in mind, it’s still Pixel Bender compiled for the Flash Player, so all the limitations in that field remain. That of course means no loops. As a result, the Pixel Bender shaders themselves are near-comical (there’s an huge unrolled double loop in the case of the Julia set, for example). I wasn’t aiming for the most optimized code either, but rather show how much difference JITB can make. So please, do take it with a grain of salt

Noteworthy, JITB actually recognizes “copy/paste unrolledness” and recompiles it to loops: very cool!

Finally

It’s been great fun to work on it, and getting to annoy Joa with feature requests Seeing how these shaders run in real time is still very thrilling to me, when you consider the normal Flash Player takes up to 2 minutes per frame! It’s not fair to compare CPU to GPU, but still…

In closing, many thanks and humble bows to Joa for inviting me and making my attempts a part of his session, as well as his hard work on Apparat! I encourage anyone to check out his work and contribute!

As promised in my previous post, I would elaborate on some of the things done for the demo. So I’m taking a quick break from work to get this post done

Marching Cubes

One thing I did was implement a marching cubes algorithm using Pixel Bender, which is a way to triangulate an isosurface in a scalar field. I had started to write up a whole explanation, but realized it was kinda pointless, as it has been covered plenty of times If you’re interested, you’re better off reading up about the subject starting here and here.

Pixel Bender

I know there’s plenty of marching cube implementations in ActionScript out there, but I haven’t seen one using Pixel Bender, so I thought I’d give it a try. I’m using it to calculate the values in the scalar field (at least on the marching cube’s grid corners), and to build the pattern ids needed for triangulation. The benefit of using Pixel Bender is that you can put in any kind of calculation that outputs scalar values, some of which you wouldn’t dare to put ActionScript through. The drawback is that it seems to have some precision problems while doing comparisons (or so it seems), so there’s some missing triangles on occasion.

No transforms, no sorting

Something I’ve realized that’s pretty neat about this algorithm is that you don’t actually need sorting. As long as you make sure the grid is aligned to the “camera” at all times, the triangulation occurs back to front and will already be correctly sorted. This of course means you can’t do any rotations on the triangles, but that’s no big deal. You can simply perform the transformation on the grid coordinates, and let the correct values be calculated for those points. Added bonus, you can do any scaling and translations together with the projection matrix in one call to Utils3D.projectVectors, annihilating the need for any calls to Matrix3D.transformVectors. Result: some extra fps.

Metaballs

Metaballs are probably the most iconic example of isosurfaces out there (bar MRI and CT imagery). It was actually my test data for the MC system, but it ended up making a sneaky appearance in the demo (which I still consider a tribute to the undisputed king of ActionScript sticky substances ).

> Metaball demo (click to change textures)

Quaternion Julia Set

Another example I did was to triangulate a quaternion Julia set, which seems pretty popular lately It definitely looks better raytraced, but I couldn’t resist! I’m using the distance estimator function to produce the grid values (see here), and an epsilon distance as the surface’s isovalue. Since things always look less crap with music, I added some for a change.

> Quaternion Julia Set Demo (might take a while to load the mp3)

Source

The marching cubes thingy, as well as the metaballs example source is up for grabs at Google Code . Enjoy! If you make any surfaces with it, I’d love to see them

A few months ago, Simo Santavirta (aka simppa.fi, aka jac) from EvoFlash invited me to do some effects for a demo they were planning to release at Assembly 2010. I hadn’t actually done anything for the demoscene before, but how could I say no; it’s Assembly, home of Second Reality! (and I do believe there was beer involved at some point). The category was “Real Wild”, which basically means “demos for any platform that can run real-time graphics”, including Flash. And… We ended first! Champagne for all!

Linkage

• Check out the demo: “Spiral Out”
• The video version
• Simo’s original  post
• The other Real Wild submissions

Demo building for the creatively challenged

Before I start yapping too much, first things first: Simo did >90% of the demo, just need to clarify that!

Okay, to continue… I did get confronted with some facts of life in that I am not an artistic person! With code as a starting point, it’s hard for me to come up with something that looks good. Feels like a wall in my brain that’s preventing me to go that way. Typically, I work the other way around and try to turn “looks” into code (like references from real life). With the abstract beauty of many demos, it had me cursing plenty of times. Luckily, Simo was able to adapt my pieces and make them look good, so I’m very proud of the end result!

From the technical side of things, here’s what I did: normal mapping on dynamic models (procedural of course), projection+shadow mapping, and marching cubes (metaballs and, I dunno, some weird tunnely thing). The metaballs are slightly tongue in cheek, and I consider them an homage to the “metaball guy”. I’ll probably get back to some of the effects in future blog posts, so I won’t delve into any detail right now. Most of them were done using Away3D Lite, preferring it over the main engine because it’s so lightweight/fast and darn easy to hack and extend. For me, it makes an excellent playground!

The demo itself was built upon the evoTinyEngine demo framework, which made it very easy to collaborate, sync, and hook effects into the main demo.

In closing

Obviously, many thanks to Simo for giving me the opportunity to join in! Too bad I couldn’t be there in person, but there were some other things I was up to during the weekend. Time to recover

Enjoy the demo!

As a continuation of an earlier post, here’s a variation of the Fresnel Pixel Bender material. Still obviously inspired by Bartek‘s Unity3D shader, it became a glass material with support for chromatic aberration (what causes the rainbow effect in refracted light). The result could come in handy in some situations, so I thought I’d share and commit it to the Away3D trunk as the GlassMaterial.

The material uses the same environment map to do (fake) refractions and reflections, so it’s not actually refracting the real scene around it. However, if you make a sphere map rendering of your scene, the effect can be convincing enough.

The demo shows a couple of default primitives and several settings of the material (such as glass colour and reflection strength), and also uses a new primitive called TorusKnot (which is a (p,q) torus knot). Consider that a tribute to a demo shown by Ralph during his presentation at FITC Amsterdam . Click to cycle through the different settings, and check the source to see how it all works.

Update: As Makc pointed out in the comments (and rightly so), the refraction looked rather icky, so I updated it that it uses linear sampling. It’s a bit slower but looks much better.

Today, we’re happy to announce the release of Away3D 2.5.0 and 3.5.0. Along with other features and changes, of which you can read more in the official announcement, it contains the new BSP/PVS support! I’ve slaved months on this, so I’m glad it’s finally made public! Keep in mind that we’re always improving it, so it’ll only get better from here So what’re all these abbreviations about?

(Warning: a long-winded post ahead. If you’re not interested in the details, stick to the official announcement!)

Binary Space Partitioning

BSP is a scene graph that divides a static scene up by splitting it recursively along planes into two half-spaces. If after splitting, the geometry in that subspace is convex, no further subdivisions are needed. In the end, we have a binary tree containing convex meshes at the leaves. What’s the use, you may ask? Well, since all the parts are convex, we don’t need to do any per-triangle sorting. By correctly traversing the tree and rendering the parts in the right order, the sorting is faster AND 100% correct. Specifically for Flash 3D, that means we don’t need to subdivide triangles to try and resolve z-fighting. Furthermore, it also greatly speeds up frustum culling and collision detection. This only works for static scenes however. There are dynamic bsp trees, but they don’t support the biggest speed-up, being the PVS.

Binary Space Partitioning on Wikipedia

The Potentially Visible Set

When we have a static BSP, we can use this data to determine which leaf nodes are visible from any given leaf node. This dataset is called the PVS (Potentially Visible Set). It allows us to prevent most of the occluded geometry from being rendered (which, usually, is most of the total geometry). Needless to say, this can lead to huge speed improvements. It’s the same technique games such as Quake, Half-Life, Unreal, … used. However, this speed improvement comes at a price (or what did you think? ), which is explained in the “Building” section of this post.

This little bastard was a pain to do. It’s the reason days turned to months of my evenings to get right, and I definitely couldn’t have done it without the help from this thesis: View Space Linking, Solid Node Compression and Binary Space Partitioning for Visibility Determination in 3D Walk-Throughs by Joel Anderson. Also, props to Joa for giving me some optimization tricks and food for thought!

Potentially Visible Set on Wikipedia

Building

Fabrice did a great job implementing the BSP/PVS builder into Prefab3D, so you can bake your lighting and generate the BSP/PVS as AWData in one programme. When compiling a model with PVS, there’s some guidelines you should keep in mind. No, let’s call them rules instead. Not obeying these could result in the build getting stuck, poor collision detection, or some random unpredictable results.

• Your model must be completely “sealed”. No leaks into “nothingness”! This is important because the PVS would try to find visible data from that leak and get stuck.
• Work on a grid. Make sure your vertices are snapped to a grid which is not too coarse. If not, the BSP splitting could create triangles that are too small, causing havoc in the form of floating point errors, too small triangles, inefficient splits, etc…
• Try to occlude a lot. Don’t make too much geometry visible from any part of the model. This speeds up the model during rendering as well as the PVS build.
• Axis-aligned walls/ceilings/floors work best. They’re fast to do calculations on and if you model somewhat smartly, they form a better “space splitter”.
• Sometimes it’s good to keep small details out of the BSP geometry, and add them using the regular addChild method as this keeps the trees cleaner and more efficient.

Finally, there’s a LOT of settings, but you should probably not have to mess too much with them unless you really know what you’re doing. Soon, we’ll have some proper tutorials explaining the nitty-gritty of building (and we might have some other tools to make things easier on you).

Check the video showing a build in action!

Demos

• The Hacienda Experiment (Beta): Fabrice and I created this for Adobe’s keynote at FITC Amsterdam in a very limited amount of time, before the BSP functionality was even finished. Lots of stress and insecurities, building the AWData exporter, etc. So, forgive us if there’s still some bugs in this version (If you get stuck, jumping out using the space bar usually does the trick )
• Bunker Demo: Made by Fabrice to illustrate how easy it is to export BSP/PVS data from Prefab3D (see video mentioned before). It also shows how fast it can run with good occlusion. (Check the source)

Thanks for sticking around for such a long post, but hey, with the amount of work gone into this, I reserve the right to bore you to death with it!

Finally, something new! But no, it’s not any of the bigger things I’ve been hinting at before. However, I’ve been hard at work at those (and other) projects for months, and ended up with tunnel vision and an annoying rut. I really needed to do something smaller again for some immediate visual fun. When Bartek showed me something on Twitter, I decided to take a step back from all the work, dig up some half-year old code and finish it asap. And so I did: that is a Fresnel Pixel Bender shader for Away3D, presented in the FresnelPBMaterial!

I’ve explained it before, but I’ll give a more general explanation this time around. Fresnel shading mimics the way light reflects off of (often translucent) surfaces such as water or glass. Looking head-on, you can see its proper surface (or see through it with refraction). However, if seen from a shallow angle, it reflects light (just imagine looking over a large body of water, or the mirage effect on the road on hot summer days).

Demos

I’ve thrown together some rather random demos to illustrate the point. There’s source for both. (A word of warning: apparently, on some FP10.1 betas, Pixel Bender causes it to throw an invalid input exception, which seems to be bogus.)

• Glass and marble head: Yes, the head appears to be floating in the middle of a cathedral of some blurry sort. Click to cycle through different property settings: glass, alien marble (I’m sure that’s a thing), and plain marble.
• Water surface: Using an animated normal map to illustrate how water could be rendered while using the fake refraction. It also shows the WaterMap util.

Another word of warning: slight implementation details might change in the next few days/weeks, but it should be nothing too dramatic!

I’ll probably be doing something more with this when I have the time, but nothing that would likely make it to the repository

Ouch! My last post has almost been 5 months ago! Realizing this, I had two options: to feign my own death until the next update, or to give some sign of (what for me passes as) life. Let’s go with the second option and see if there’s anything useful I can pass on anyway.

Not having posted in such a long time doesn’t mean I’ve been slacking off, though. Quite the contrary; the past months have seen some of the most intense coding sessions I’ve went through in a long time. First a 3-month non-stop coding spree for Away3D, followed by continued development on Farbe. Hopefully, the result of both will be available in the (very?) near future, along with some other updates!

Now, instead of giving you demos or code, here’s some other things to help you develop.

(and no, I won’t be sharing any thoughts on the future of Flash)

Books!

Books form such an important part of my development process and knowledge, I’m a bit surprised myself that I haven’t really shared any titles here. Trying to ammend the situation, here’s some of the more influential ones I’ve read since I’ve started this blog (leaving out more general ones concerning design patterns etc):

• The Pragmatic Programmer, by Andrew Hunt and David Thomas: it’s a classic, and you should already have read it. So why am I still listing it? Because it often seems many Flash platform programmers stick to ActionScript/Flex-related books. Here’s a book that changes the way you think about your code, be it in form of design principles, tools or workflow. It’s a rather short read, so no excuses!
• Elements of Programming,  by Alexander Stepanov and Paul McJones: I found this one on a random trip to the bookstore. Approaching algorithms and data structures in a very formal and mathematical way, it forces you to – again - think differently about your code, but on a lower level (mainly concerning algorithms using data structures). What I love about this type of books is that you suddenly realize how differently you start tackling certain algorithmic problems. Perhaps not a book for everyone, but an interesting read nonetheless.
• 3D Game Engine Design, by David H. Eberly: (Thanks to Ralph for suggesting it way back ). “The Morgan Kaufmann Series in Interactive 3D Technology” has quite a few good books, but to me, this is the most valuable. Of all books I had at the time, this one taught me most about many 3D engine concepts. During the development of Wick3d, I couldn’t do without it.

IDE – IntelliJ

I’ve recently started using IntelliJ IDEA 9 for ActionScript and Flex development, after hearing Joa and the Flash Bum raving about it. Coming from Eclipse (Flex Builder and FDT), it took a while to get used to, but now I have I can’t imagine returning to Eclipse at all. I have been raving about it in real life, much to the annoyance of my co-workers, so it would be wrong of me not to mention it here.  The refactoring, template and code generation support is simply unmatched, there’s a very useful and clean UML diagram view, properly integrated Maven support, … AND I get to annoy the Java/back-end team directly inside the IDE

To think I was once had to code C in MS-Dos Edit, how times have changed…

Edit: The Flashbum just wrote an article on InsideRIA about how to get started. Read all about it!

That’s about it. I hope this sign of life is useful to anyone after all! If not, I promise there will be more real updates soon!

It’s no secret that I like graphics. It’s the main reason why I play video games and it’s the main reason I got into programming. So obviously I was delighted to be invited and to join to the Away3D team last month. Inspired by my earlier Stok3d project (now on the backburner for a bit), I set off to create similar normal mapped pixel shaders, this time in full 3D. After some rough first patches (Stok3d was pretty simple since it only used DisplayObjects, flat planes), things have luckily shaped up, leading up to the first release!

The current state is a dot-release (3.4.2), so the exact implementation might still change while we’re working towards a shiny new 3.5.

The new shaders

So what is the difference with the previous shaders? Using Pixel Bender, the shading is calculated for every pixel in the texture, resulting in much more detailed and realistic lighting or reflections. Each shader requires an object-space normal map, which you can use to add detailed shading information without increasing the polycount of the mesh.

The shaders come in three flavors: environment map shaders, and single- and multi-pass shaders. Single-pass shaders take one PointLight3D and any AmbientLight3D on the scene to calculate the lighting, whereas multi-pass takes any number of lights of any type (AmbientLight3D, DirectionalLight3D, and AmbientLight3D). Important to note, tho, is that every light adds another pass and will be slower. Of course, if you’re only using 1 light, always use single pass.

Check out the following classes in Google Code:

• SphericEnvMapPBMaterial:  spheric environment mapping
• Cubic EnvMapPBMaterial: cubic environment mapping

• DiffusePBMaterial: Single-pass, adds diffuse lighting to the texture
• PhongPBMaterial: Single-pass, adds diffuse lighting and specular highlights, with support for specular maps
• SpecularPBMaterial: Single-pass, adds only highlights – can be used in combination with Prefab3D‘s prebaked lighting to create view-dependent specular highlights

• DiffuseMultiPassMaterial: Diffuse shading with multiple light sources
• PhongMultiPassMaterial: Phong shading with multiple light sources

Demos

Enough explanations, time for some demos! Right-click to view source:

• Cubic environment mapping
• Single pass shading: showing how a detailed normal map and specular map can enrich the shading
• Multi pass shading: using 3 lights (directional moonlight, firefly, and a flickering flame) and ambient

That’s it, enjoy! Feel free to drop by the mailing list for questions or read the official announcement! For now, I need to get some sleep before Flash on the Beach kicks off

The summer holidays aren’t even over yet and September already seems like it’s going to be a great month! Apart from the fact that I’m going to Flash on the Beach, there’s also something for us Belgians that’s a bit closer to home: the annual Adobe User Group meeting & barbeque on September 12th! And as it happens, I’ll be speaking there alongside much more impressive names as Seb Lee-Delisle and Niqui Merret

What to expect? Here’s my session description:

“In his session, David will walk through some of the experiments and projects he has done over the past year while elaborating on the concepts behind them. Without getting too technical, he will explain his motives, his love affair with Pixel Bender and the big looming question: “Mmkay, but what’s the use?”. Using experimentation as the main thread, he will try to show how going off the beaten path can take you to sweet little places you didn’t expect, and finally, offer a sneak-peak at some of the more useful projects he ended up doing as a result.”

In short, since it’s my first public session, I figured I would give a bird’s view of my philosophy concerning after-work development, and show some cool things I have in the pipeline at the moment (so even if you follow this blog, there’ll be something new to look at ).

Don’t forget to register here if you want to come (and gloat) – and if you do, I’ll see you there!

Since I started playing around with Pixel Bender in Flash, I’ve been trying out some different approaches here and there and learned a thing or two on performance optimizations (and quirks). As many people use PB specifically for its performance, and not much has been written on the subject, I thought I’d share my experiences and back them up with some benchmarks. Some of the things here are pretty obvious, yet others can be surprising and even frustrating.

Remember that this concerns Shaders in Flash Player, not Photoshop or After Effects, and that results could change in future versions. All benchmarks were performed on my crummy pc (AMD Athlon 64 X2 Dual, 2.21Ghz, 2GB Ram, Win XP), using 500×500 data with 4 channels, each performing 10 consecutive kernel executions. The kernel itself is just a read, a multiplication, a division, and a sqrt. ShaderJobs are performed synchronously.

Let’s get the obvious out of the way first (I won’t go into common sense optimalizations too much).

Well, duh!

• Use 4 channels only if necessary. No transparency? Ditch it.
• Precalculate recurring constant calculations in Flash and pass them as parameters (such as width*height). Sure, it makes the “interface” of your Kernel potentially harder to read, but since Flash doesn’t support dependents (I hope it will some day), this should be a no-brainer if performance is really important.
• If only a part of a BitmapData needs to be processed, isolate it into a new BitmapData using copyPixels. Even when using applyFilter, sourceRect is buggy.

Told you it was obvious :p Now, some better ones.

Use ShaderJob, not ApplyFilter

• ShaderJob (on BitmapData) benchmark: 92-99ms
• ApplyFilter benchmark: 104-109ms
• ShaderJob ~ 10% faster

BitmapData is faster than ByteArray is faster than Vector.<Number> !

I’ve seen (and been guilty of) a lot of copying BitmapData into a Vector to harness “the power of Vector”. But look at this:

• ShaderJob on BitmapData: 92-99ms
• ShaderJob on ByteArray: 147-172ms
• ShaderJob on Vector.<Number>: 167-192ms
• BitmapData is ~40% faster than ByteArray
• BitmapData is ~47% faster than Vector.<Number>!!

Use BitmapData unless you have no other choice, or if complete floating point precision is important.

Conditionals are expensive!

This one annoys me quite a bit. Imagine you’re doing some calculations that you don’t need to do when alpha == 0 (which, as it happens, is usually the case). It can be a good idea to do them anyway in favour of dropping the alpha == 0 check. For the benchmark, I used values that had alpha set to 0 for about half of the data! Compare results to the previous benchmark.

• BitmapData: 134-192ms – ~47% speed loss!!!
• ByteArray: 147-172ms – ~22% speed loss
• Vector: 192-213ms – ~27% speed loss

In practice, test a version with and one without conditional. The results vary heavily depending on how many times calculations are omitted, and how many calculations are otherwise performed. Still, with half the (although slightly trivial) calculations omitted in this case, it’s stupefying that there’s so much increase in execution speed.

Do not use the input as the output

When using a ShaderJob or ApplyFilter, don’t use the same BitmapData/ByteArray/Vector instance that functions as the source. If you need iteration, you’re better of swapping two buffers. What happens is that Flash Player will need to make a temporary copy of the source, which slows things down.

Edit: The results here were compared to the normal ShaderJob test, while they’re using the alpha test. Percentages have been updated

• BitmapData: 207-218ms – ~30% speed loss
• ByteArray: 256-271ms – ~65% speed loss
• Vector: 276-293ms – ~40% speed loss

Update: Asynchronous ShaderJob

I just tested it, and the results indicate that asynchronous calls (waitForCompletion=false) are slower than synchronous calls. I suppose that’s mainly because of the event handling flow. Another thing I tested was to run 2 asynchronous calls with data of half the size, but it seems only 1 asynchronous ShaderJob can be started at the same time.

That’s it, see for yourself!

In closing, I’ll mention something I usually do but doesn’t seem to have any effect (it’s actually a habit from ActionScript). When reading from the same coordinate multiple times, I often store outCoord() in a variable and use that in the sample function. Well, I tested it, and it doesn’t have any impact at all

That’s it, at least for now, I hope it’s helpful! Check the benchmark and its source (the source is in fact pretty ugly, but does the trick). I’d be happy to know what kind of results other hardware yields.