Flash Player 10 came with a set of new classes to load and use Pixel Bender files in our Flash applications. In this tutorial we'll take a look at Pixel bender syntax and how to write your own Pixel Bender files. We will also see how to load Pixel Bender files in Flash and use them in different ways.

First if you still did not download the Pixel Bender Toolkit you should do so here. The Pixel Bender Toolkit comes with a set of Pixel Bender filters so feel free to study them to get a better understanding of Pixel Bender syntax. You can also search the web to find additional Pixel Bender filters.

In this tutorial we'll create some simple filters with the Pixel Bender Toolkit and later on we'll use some existing ones so let's get started.

Start the Pixel Bender toolkit and choose "New Kernel", a complete new working kernel file is automaticaly created for you. You can already run this filter, you'll probably be prompted to load a picture and after doing so, the filter will be running. Of course the only thing this filter is doing so far is copy each pixels from the base image into the output image so that's not that usefull but let's take a look anyway. First you have the kernel description:

kernel NewFilter
< namespace : "Your Namespace";
vendor : "Your Vendor";
version : 1;
description : "your description";
>

You can give a name to your filter here as well as other information like the version of this filter, a description and so on. Next:

input image4 src;
output pixel4 dst;

You specify an input of type image4 (4 channels) and a name "src". The filter will use this input as a base. You also specify an output of type pixel4 (4 channels) and a name "dst". The filter will pass the result of its calculation to this ouput. Then:

void
evaluatePixel()
{
dst = sampleNearest(src,outCoord());
}

This function (evaluatePixel) will run once for each element (pixel) in your input data. The sampleNearest method is returning a pixel value and this pixel value is assigned to the "dst" output. The sampleNearest method takes a source (in our case an image4 source), and a set of coordinate (outCoord() returns the coordinates of the current element or pixel). If you pass an image4 you get back a pixel4 value, if you were to pass an image3, you would get a pixel3 and so on. The number determines how many values you get or have to set for example:

float myfloat = 1.0;//one value
float2 myotherfoat = float2(1.0,1.0);//two values
float3 mythreefoat = float3(1.0,1.0,1.0);//three values
float4 myfourfoat = float4(1.0,1.0,1.0,1.0);//four values

Casting is possible of course if type are compatible like from int to float for example. Obviously if you try to pass a pixel 3 to an image4 you'll get yourself an error so just be aware of that. Let's modify our code so our output looks different:

dst = pixel4(sin(float4(sampleNearest(src,outCoord()))));

You should get a slight alteration of your picture lightness. You probably know the sin method already and what it does. Here we casted our sampleNearest result into a float4 (so we casted from pixel4 to float4) so we could apply the sin method, then of course we casted back to pixel4 so we could assign the result into our output. Already you can experiment with the numerous math methods and see what result you get. try this one for example:

dst = pixel4(sqrt(float4(sampleNearest(src,outCoord()))));

Moreover we could separate each pixels and apply some calculation on each one of them:

pixel4 temp = sampleNearest(src,outCoord());
dst = temp;

Back to the beginning but here we have our pixels stored in a variable so we can access them individually so let's do that now:

pixel4 temp = sampleNearest(src,outCoord());
temp.r = 0.0;
dst = temp;

Here we basically turned off the red channel so all red is gone from our picture. You could apply numerous math operation on these pixels and get some amazing result. Of course your level in math will determine your ability to create complex filters but this is for you to discover. Now it's time to take a look at what we can do with Pixel Bender filters in Flash and as we go we'll look at more Pixel Bender code of course.

 Even though Pixel Bender filters are meant and used to alter pixels, what they really do is applying math operations to numbers. When we use pixel bender filters in Flash, we can actually use them to process numbers or to alter an image. We'll start by looking at an example with number processing and then we'll move on to pixel manipulation.

To be able to load a Pixel Bender file in Flash, it needs first to be compiled in a .pbj format and we can do so by selecting "Export filter for Flash Player" in our Pixel Bender Toolkit. I created a simple Pixel Bender filter that will allow us to manipulate numbers, here is how this looks:

<languageVersion : 1.0;>

kernel NewFilter
< namespace : "red divider";
vendor : "aswebcreations.com";
version : 1;
description : "simple dividing on red channel";
>
{
input image1 src;
output pixel3 dst;

void
evaluatePixel()
{
pixel1 temp = sample(src,outCoord());
pixel1 divided = temp/2.0;
pixel1 squared = sqrt(temp);
dst = pixel3(temp,divided,squared);
}
}

Now you can see that our image input is of type 1 which means it's expecting one number instead of 3 for a regular pixel (rgb) or 4 for rgba pixels. Our output is pixel3 based and this is a minimum for an output in Pixel Bender (as far as I know). Then in the evaluatePixel() method we get our number and store it in a variable:

pixel1 temp = sample(src,outCoord());

Then we store the result of dividing by 2 in another variable:

pixel1 divided = temp/2.0;

And finaly the result of squaring our number in another variable:

pixel1 squared = sqrt(temp);

We of course then pass this result to our output:

dst = pixel3(temp,divided,squared);

You can of course apply this filter to a picture but its real purpose is to process numbers so let's get started with our actionscript code now:

var loader:URLLoader = new URLLoader();
    loader.dataFormat = URLLoaderDataFormat.BINARY;
    loader.addEventListener(Event.COMPLETE, onLoadComplete);
    loader.load(new URLRequest("divider.pbj"));
     
function onLoadComplete(event:Event):void {

}

This loads our filter in Flash what we need now is create an instance of the Shader class and get our filter data ready:

var shader:Shader;
function onLoadComplete(event:Event):void {
shader = new Shader(loader.data);
}

Our filter is now ready to be used! There's 3 data types we can use as Shader input and ouput, BitmapData, byteArray, and Vector.<Number> (note that some filters do not require inputs). I've seen already examples on the web about processing numbers using ByteArray so I thought I would use Vector for this tutorial instead. I'm going to create a Vector and populate it with random numbers then set it as the shader input. Then I'll create another vector that I'll use to store the results coming from our filter (Note that it's possible to use the same object as input and output in which case results are building up on preceding results which can be interesting for bitmap alteration for example).

function onLoadComplete(event:Event):void {
shader = new Shader(loader.data);
var vector:Vector.<Number> = new Vector.<Number>();
for(var i:Number = 0; i<9; i++){
    vector.push(Math.random()*500);   
    }
shader.data.src.width = 3;
shader.data.src.height = 3;
shader.data.src.input = vector;
var vector2:Vector.<Number> = new Vector.<Number>();
var shader_job:ShaderJob = new ShaderJob(shader, vector2, 3, 3);
shader_job.start(true);
var txt:TextField = new TextField();
txt.wordWrap = true;
txt.width = stage.stageWidth;
txt.height = stage.stageHeight;
addChild(txt);
for(i = 0; i<vector2.length; i+=3){
    txt.appendText((vector2[i]+" divided by 2 = "+vector2[i+1]+" and squared = "+vector2[i+2]+"\n"));
    }
}

Here is the result:



Let's look at the code:

var vector:Vector.<Number> = new Vector.<Number>();
for(var i:Number = 0; i<9; i++){
vector.push(Math.random()*500);
}
shader.data.src.width = 3;
shader.data.src.height = 3;
shader.data.src.input = vector;

Here we simply create a Vector.<Number> and populate it with some random numbers (9 numbers in this case). Then we pass some values to our shader input called "src" (defined in the pixel bender). We indicate the size of the input with src.width = 3 and src.height = 3 (which means 9) and finally pass our Vector as the input: src.input = vector.

var vector2:Vector.<Number> = new Vector.<Number>();
var shader_job:ShaderJob = new ShaderJob(shader, vector2, 3, 3);
shader_job.start(true);

We create a new Vector and pass it as the output in the ShaderJob instance. We pass along the Shader we want to use of course and the size of the ouput we are expecting (3,3). The rest of the code creates a text field to display our result. Then:

for(i = 0; i<vector2.length; i+=3){
txt.appendText((vector2[i]+" divided by 2 = "+vector2[i+1]+" and squared = "+vector2[i+2]+"\n"));
}

That's right, we didn't get 9 numbers back but 27! Remember our output is of type pixel3 so that means 3 numbers for each number that we process!

A ShaderJob instance uses the GPU to process the Pixel Bender filter so this leaves the CPU take care of normal stuff in our Flash applications while the Pixel Bender is running. This creates a multithreaded Flash application! Passing true to the start method makes Flash wait for the result to get back while passing false (or nothing) makes the results asynchronous. You can then add an event listener to your ShaderJob instance and handle the results in your event handler. Now we could make our filter more dynamic by adding a parameter:

input image1 src;
output pixel3 dst;
parameter float div;

pixel1 divided = temp/div;//in the evaluatePixel() method

We now added a parameter in our filter which allows us to change on the fly some value like here the number we are dividing by (defaults to zero). We would rewrite our actionscript code that way:

shader.data.src.width = 3;
shader.data.src.height = 3;
shader.data.src.input = vector;
shader.data.div.value = [6];

Note the array style value. All values passed to parameters must be in array form. Now if we were to save our filter and add this new line of code then our numbers would be divided by 6. We could also of course tween this value and get our results back in real time. In Pixel Bender we can also provide default values, minimum values and maximum values to our parameters which we can then retrieve in our Flash application. But let's leave this for now and start experimenting with bitmap manipulation.

Here is where the fun really begins. We can very easily apply a pixel Bender filter to a BitmapData which will alter all pixels in it or we can also use a Pixel bender filter as a filter like you would use any other Flash built-in filter. Let's see this in action, I added a BitmapData in the library and loaded a different filter:

var shader:Shader;   
var bitmap:Bitmap = new Bitmap(new Autumn(0,0));
addChild(bitmap);
function onLoadComplete(event:Event):void {
shader = new Shader(loader.data);
stage.addEventListener(MouseEvent.CLICK, addBender);
}
function addBender(e:MouseEvent):void{
shader.data.src.input = new Autumn(0,0);//set the input
var shaderjob:ShaderJob = new ShaderJob(shader, bitmap.bitmapData);//apply the filter on the bitmapdata of our bitmap
shaderjob.start();
}

And here is the result, click once to apply the filter:



This alters directly the BitmapData pixels. Another way is to create a ShaderFilter which you can then apply to any DisplayObject:

var shaderfilter:ShaderFilter;
function onLoadComplete(event:Event):void {
shader = new Shader(loader.data);
shaderfilter = new ShaderFilter(shader)
stage.addEventListener(MouseEvent.CLICK, addBender);
}
function addBender(e:MouseEvent):void{
bitmap.filters = [shaderfilter];
}

Here is the result:


Now that's not a lot of code. We could now tween properties using either tweens or timer or enterframe events and get quite impressive results. I'll leave that for you to experiment and come out with your own solutions. But to get to the point I wrote a little class (see zip file) that does just that so let's try on some filters here:









The code for all this is quite simple and it's about the same for all these examples, only the property being animated changes. You can just download all the files and play with all this. Just remember that the property must match the property defined in the Pixel Bender.

import ASWC.*;

var shader:ShaderExtended = new ShaderExtended("cone.pbj");
shader.addEventListener(ShaderExtended.SHADER_READY, get_filter);
var bitmap:Bitmap = new Bitmap(new Tree(0,0));
addChild(bitmap);
function get_filter(e:Event):void{
stage.addEventListener(MouseEvent.CLICK, addBender);
}
function addBender(e:MouseEvent):void{
shader.setInput({src:new Tree(0,0)});
shader.animate(30, null, { offset:[10,0]}, bitmap.bitmapData, false);
}

That's all for this introduction to Pixel Bender and Shader. Contact me if you have any questions and don't hesitate to leave a comment!