Just wanted to let you all know that I will be presenting Microsoft Kinect and Molehill at this upcoming 360Flex, April 10-13. Here is the description:

“The Microsoft Kinect, with its stereoscopic, infrared wizardry, has given us a fascinating tool for rich interaction design. For the first time, a high-quality, depth-sensing camera is widely available at a price many people can afford. While this is exciting, equally so is Adobe’s new Molehill API, which takes advantage of GPU acceleration, allowing Flash direct access to DirectX and OpenGL.

The Kinect and Molehill are a match made in heaven. Come learn how to get the Kinect up and running on your machine, while also exposing it to Flash. Also learn the basics of Molehill and watch how Molehill and Kinect can work together. ”

Register here!

Hey all, I will be giving a talk on UX: The Art of Interaction at Flash and the City coming up in June! Here is the description:

UX: The Art of Interaction

Explore the driving forces behind User Experience Design. Join Leonard Souza, an Experience Architect with EffectiveUI, for a in-depth session aimed at delving into the finer points of UX for beginners and seasoned veterans alike. Learn the practical application of UX fundamentals while witnessing them first-hand in examples backed by theory. So come on down with an open-mind and a lot of questions; this one is going to be fun!

Topics explored:

Promoting Optimum Experience
Visual Psychology
Powerful Game Mechanics
Exploiting Cognitive Associations
Interactive Storytelling
and more…

This conference is the bees knees when it comes to Flash, plus it’s in NY. Register here: http://fatc11.eventbrite.com/

Here is a link to the GitHub repo for my presentation today.

And another for a direct download of a zip with the FXP and Keynote slides.

Article to follow, so stay tuned!

I’ll be speaking at 360Flex, which is in Washington D.C., September 19-22. If you’ve never experienced a 360, then you are missing out on one of the best Flex/Flash conferences. The level of training is insane, as well as the amount of networking opportunities. This will be my third time speaking at this conference, and I have to say my own career has been propelled by the people I was fortunate enough to hang out with. My presentation will revolve around some awesome Flex effects, which include some pixel bender magic. The description is below. Sign up if you are interested!

Flexerific Visual Effects

Flex has some amazing built-in visual effects and filters, which are probably one of the primary reasons to use the framework over other RIA platforms. As it turns out they are relatively easy to create. This session will dive into the drawing API, the base effect/filter classes and Pixel Bender. You’ll learn how to build them from the ground up in a way that is reusable for yourself and other interaction designers. Need to make a group turn into flickering static? Or shift the RGB of an image? How about create dynamic heatmaps? Then come down and learn!

SOMETIMES GOOD JUST HAPPENS

I gotta say, Flex 4 has come with some serious improvements. This is notable considering how wonderful Flex 3 was. The Flex SDK team keep bringing on the good like it aint’ no thang. After learning Spark skinning and then delving into all that is Spark, I became smitten with their new philosophy toward the Flex component architecture. It is clean, decoupled and a joy to work with. Out of all the improvements however, I keep finding myself playing around with the LayoutBase class.

You see, layouts are no longer coupled to a container such as mx::canvas. They are freely interchangeable as long as they extend LayoutBase. This presents some exciting opportunities for guys like me who love to break from typical layout conventions and play with moving elements in interesting ways. Want a 3D carousel? Just create a 3DCarousel class which extends LayoutBase and slap that puppy in there. BOOM! You have yourself a 3D Carousel. But the best part is that you can swap those layouts in realtime, so if that carousel just isn’t scratching that itch, slap a SolitaireLayout in there. BOOM! Simple as pie.

EXAMPLES

The two examples I’m showing today are below. The first is a modified version of the example RobotLegs Flickr gallery app created by Joel Hooks, and when I say modified, I mean simply swapping the layout for the datagroup holding the image item renderers and making a few changes to its mediator. The second is an app I created to demonstrate the power of the layout manager and hopefully inspire some folks to see just how far the layout manager can go. Grab the Git repositories; the first is here and the second here.

Flickr Gallery Layout

Wait a minute for the images to load. Once all of the thumbnails are visible, click on one and watch the magic happen. All of the animation and grid layout are taking place in the Layout class.

Image Analysis Layout

The elements on the right are being laid out by analyzing the thumbnail image on the left. Click the drop down for some more images or add your own with the input beneath. Feel free to play with the Layout properties, such as grid density and contrast threshold. Even use the scratch pad to draw out your own Layout!

SPARK LAYOUT

So the above examples extend the Spark LayoutBase class. It’s a rather simple Spark class, aside from the methods to manage scrolling and drag/dropping. To get started, you really only need to override updateDisplayList(), and if you want it to play nice with other components then measure() as well (NOTE: my image layout component does not override measure). The Flickr Gallery example actually takes the existing s:TileLayout class and just adds an animation method to it, while also tying it to a specific item renderer type (yes, I should have used an interface so that the class is not tightly coupled, but I didn’t). So if you take a look at updateDisplayList(), it’s the same as the native s:TileLayout’s.

I’m almost 100% sure anything can be improved with a little bit of qualified animation – I use “qualified” because bad animation can and will ruin all that is great. However, if used appropriately, animation has the ability to inject a tremendous amount of character into a design. I will be exploring this concept in a future blog post, so stay tuned. Anyways, I wanted to make my grid act a little crazy, which is why I added a few methods. They are below. Check out Christophe Coenraet’s tutorial on adding animation to a Layout for more info.

/* Animation Code */

private var _flipEffects:Parallel;
public var isPieces:Boolean = true;
private var _oldHorizontalGap:int;
private var _oldVerticalGap:int;

public function animateFlip(vec:Vector.<BitmapData> = null):void
{
	if (_flipEffects != null && _flipEffects.isPlaying) return;
	target.autoLayout = false; // Do not call updateDisplayList()

	_flipEffects = new Parallel();
	var numElements:int = target.numElements;
	if (numElements == 0) return;

	var itemRenderer:GalleryImageThumbnailItemRenderer;
	var flipEffect:AnimateProperty;

	var delayCount:int = 0;
	var reduceCount:int = 0;
	for (var i:int = 0; i < requestedColumnCount; i++)
	{
		for (var j:int = 0; j < requestedRowCount; j++)
		{
			try
			{
				delayCount += 5;
				itemRenderer = GalleryImageThumbnailItemRenderer(target.getElementAt((i*(requestedRowCount))+j));
				// This doesn't work! Boo! Neither does translating the matrix3d manually! It also kills all mouse events on the renderer, lame.
				// itemRenderer.maintainProjectionCenter = true;
				itemRenderer.imagePiece = vec[(i*(requestedRowCount))+j];
				flipEffect = new AnimateProperty(itemRenderer);
				flipEffect.startDelay = delayCount;
				flipEffect.duration = 400;
				flipEffect.property = "rotationX";
				flipEffect.fromValue = 0;
				flipEffect.toValue = 360;
				flipEffect.addEventListener(TweenEvent.TWEEN_UPDATE, function f(event:TweenEvent):void
				{
					var item:GalleryImageThumbnailItemRenderer = GalleryImageThumbnailItemRenderer(event.target.target);
					if (event.value >= 180)
					{
						showPieces(item, isPieces);
						AnimateProperty(event.currentTarget).removeEventListener(TweenEvent.TWEEN_UPDATE, f);
					}
				});
				flipEffect.addEventListener(TweenEvent.TWEEN_END, function f(event:TweenEvent):void
				{
					var item:GalleryImageThumbnailItemRenderer = GalleryImageThumbnailItemRenderer(event.target.target);
					if (item.isPiece == isPieces) showPieces(item, isPieces);
				});

				// Flip Effects

				_flipEffects.addChild(flipEffect);
				_flipEffects.addEventListener(EffectEvent.EFFECT_START, function f(event:EffectEvent):void
				{
					if (!isPieces)
					{
						_oldHorizontalGap = horizontalGap;
						_oldVerticalGap = verticalGap;
					}
				});
				_flipEffects.addEventListener(EffectEvent.EFFECT_END, tweenGap);
				_flipEffects.addEventListener(EffectEvent.EFFECT_END, function f():void
				{
					target.autoLayout = true;
				});
			}
			catch (error:Error)
			{
				trace(error.message);
			}
		}
		if (reduceCount == 0) reduceCount = delayCount * .7;
		delayCount -= reduceCount;
	}

	_flipEffects.play();
}

protected function showPieces(item:GalleryImageThumbnailItemRenderer, bool:Boolean):void
{
	bool ? item.showThumb() : item.showPiece();
}

protected function tweenGap(event:EffectEvent):void
{
	var tween:GTween;
	if (!isPieces)
		tween = new GTween(this, .2, {horizontalGap: 0, verticalGap: 0}, {ease:Back.easeIn});
	else
		tween = new GTween(this, .2, {horizontalGap: _oldHorizontalGap, verticalGap: _oldVerticalGap}, {ease:Back.easeOut});
}

Essentially the above code will flip every grid item in a falloff sequence making it shimmer like a ripple. I can then call this animation method from outside the layout class. So when you click on a tile, which is an Image that needs to be loaded, I fire off animateFlip after the load is complete and replace each tile with pieces of the larger image and pull them together. This all happens in the view’s mediator, which is where the following code exists.

protected function selectImage(image:GalleryImage):void
{
	galleryView.animatedLayout.isPieces = galleryView.animatedLayout.isPieces ? false : true;

	if (!galleryView.animatedLayout.isPieces)
		dispatch(new GallerySearchEvent(GallerySearchEvent.SEARCH_NOT_AVAILABLE));
	else
		dispatch(new GallerySearchEvent(GallerySearchEvent.SEARCH_AVAILABLE));

	progress.alpha = 0;
	PopUpManager.addPopUp(progress, galleryView, false);
	PopUpManager.centerPopUp(progress);

	var toY:int = progress.y;
	progress.y = -progress.height;

	var tween1:GTween = new GTween(progress, .5, {alpha:1}, {ease:Sine.easeOut});
	var tween2:GTween = new GTween(progress, .5, {y:toY}, {ease:Back.easeOut});

	var loader:Loader = new Loader();
	loader.contentLoaderInfo.addEventListener(Event.COMPLETE,
		function f(event:Event):void
		{
			selectedImageVector = divideAndConquer(Bitmap(event.target.content));
			var tween1:GTween = new GTween(progress, .5, {alpha:0}, {ease:Sine.easeIn});
			var tween2:GTween = new GTween(progress, .5, {y:galleryView.height}, {ease:Back.easeIn});
			tween2.dispatchEvents = true;
			tween2.addEventListener(Event.COMPLETE, function f():void
			{
				galleryView.animatedLayout.animateFlip(selectedImageVector);
			});
		});
	loader.contentLoaderInfo.addEventListener(IOErrorEvent.IO_ERROR,
		function f(event:IOErrorEvent):void
		{
			var tween1:GTween = new GTween(progress, .5, {alpha:0}, {ease:Sine.easeIn});
			var tween2:GTween = new GTween(progress, .5, {y:galleryView.height}, {ease:Back.easeIn});
			Alert.show("Error loading image!");
		});

	loader.load(new URLRequest(image.URL));
}

private function divideAndConquer(source:Bitmap, rectWidth:int = 30, rectHeight:int = 30):Vector.<BitmapData>
{
	var columns:int = galleryView.animatedLayout.requestedColumnCount;
	var rows:int = galleryView.animatedLayout.requestedRowCount;

	var background:BitmapData = new BitmapData(galleryView.dgContainer.width, galleryView.dgContainer.height, false, 0x000000);
	var bgActualHeight:Number = background.height - (galleryView.animatedLayout.verticalGap * galleryView.animatedLayout.requestedRowCount) + galleryView.animatedLayout.verticalGap;
	var bgActualWidth:Number = background.width - (galleryView.animatedLayout.horizontalGap * galleryView.animatedLayout.requestedColumnCount) + galleryView.animatedLayout.horizontalGap;

	var scaleFactor:Number = getScaleFactor(source, bgActualWidth, bgActualHeight);
	var matrix:Matrix = new Matrix(scaleFactor, 0, 0, scaleFactor, (bgActualWidth/2 - (source.width*scaleFactor)/2), (bgActualHeight/2 - (source.height*scaleFactor)/2));
	background.draw(source.bitmapData, matrix); 

	var bitmapVector:Vector.<BitmapData> = new Vector.<BitmapData>();
	for (var i:int = 0; i < rows; i++)
	{
		for (var j:int = 0; j < columns; j++)
		{
			var rect:Rectangle = new Rectangle(j * rectWidth, i * rectHeight, rectWidth, rectHeight);
			var bm:BitmapData = new BitmapData(rectWidth, rectHeight, false, 0x000000);
			bm.copyPixels(background, rect, new Point());
			bitmapVector.push(bm);
		}
	}
	return bitmapVector;
}

<s:DataGroup
	id="dgThumbnails"
	dataProvider="{this.dataProvider}"
	itemRenderer="org.robotlegs.demos.imagegallery.views.components.renderers.GalleryImageThumbnailItemRenderer">
	<s:layout>
		<layouts:GalleryTileLayout
			id="animatedLayout"
			verticalGap="5" horizontalGap="5" requestedColumnCount="20" requestedRowCount="10" />
	</s:layout>
</s:DataGroup>

And voila, you now have an animated grid that flips as expected!

IMAGE ANALYSIS LAYOUT

My second example, the ImageLayout class, does not blatantly steal any code from the existing Spark Layout classes. It is a relatively small class, with two methods who do most of the work: updateDisplayList() and arrangeByImage(). The ImageLayout takes a bitmap as a source property and when the container’s display list is invalidated, it will chop up the bitmap source into a grid, then lay its element according to the brightness values of the grid squares. For a finer analysis, increase the grid density or reverse it for the opposite effect. Not only does it place the elements on the X and Y axis, but will place on the Z according to the brightness intensity. This creates a bumpmap-like effect, similar to what you would see in 3D rendering though much more primitive.

protected function arrangeByImage():void
{
	if (!target || !_source) return;

	_layoutVector = new Vector.<Vector3D>();

	// scale the image to the width and height of the target to interpolate the points
	var nonScaledWidth:int = _source.width / _source.scaleX;
	var nonScaledHeight:int = _source.height / _source.scaleY;

	_source.scaleX = target.width / nonScaledWidth;
	_source.scaleY = target.height / nonScaledHeight;

	var scaledSource:BitmapData = new BitmapData(_source.width, _source.height, true, 0x000000);
	scaledSource.draw(_source, _source.transform.matrix);

	var rectWidth:Number = _source.width / gridInterval;
	var rectHeight:Number = _source.height / gridInterval;

	var columnCount:Number = _source.width / rectWidth;
	var rowCount:Number = _source.height / rectHeight;

	var i:Number;
	var j:Number;

	var reqElements:int = 0;

	for (i = 0; i < rowCount; i++)
	{
		for (j = 0; j < columnCount; j++)
		{
			var rect:Rectangle = new Rectangle(j * rectWidth, i * rectHeight, rectWidth, rectHeight);
			var bm:BitmapData = new BitmapData(rectWidth, rectHeight, false, 0x000000);
			bm.copyPixels(scaledSource, rect, new Point(0, 0));

			var brightnessPercentage:Number = averageColor(bm)/0xFFFFFF;
			if (brightnessPercentage * 100 >= contrastThreshold)
			{
				_layoutVector.push(new Vector3D(j * rectWidth, i * rectHeight, brightnessPercentage * depthFactor));
				reqElements++;
			}
			else
			{
				_layoutVector.push(null);
			}

			bm.dispose();
		}
	}

	requiredElements = reqElements;
	dispatchEvent(new ImageLayoutElementEvent(ImageLayoutElementEvent.ELEMENT_CHANGE, requiredElements, target.numElements));
	scaledSource.dispose();
}

/*
** Overrides
*/

override public function updateDisplayList(w:Number, h:Number):void
{
	if (!target || !_source || _source.height + _source.width < 2) return;
	super.updateDisplayList(w, h);

	arrangeByImage();

	var el:ILayoutElement;
	var numElements:uint = target.numElements;
	var matrix:Matrix3D;
	var i:int = 0;

	if (!_layoutVector)
	{
		for (i = 0; i < numElements; i++)
		{
			el = target.getElementAt(i);
			if (!el || !el.includeInLayout) continue;

			matrix = new Matrix3D();
			matrix.appendTranslation(0, 0, 0);
			el.setLayoutMatrix3D(matrix, false);
		}
		return;
	}

	var vec3D:Vector3D;
	var elementCounter:int = 0;
	for (i = 0; i < _layoutVector.length; i++)
	{
		try
		{
			el = target.getElementAt(elementCounter);
			if (elementCounter >= numElements) throw RangeError("Not enough elements");
			vec3D = _layoutVector[i];
			if (vec3D != null)
			{

				if (!el || !el.includeInLayout) continue;
				UIComponent(el).visible = true;
				matrix = el.getLayoutMatrix3D();

				if (animate)
				{
					var originVector:Vector.<Vector3D> = matrix.decompose();
					//originVector[0] = new Vector3D(Math.random()*animationStrength, Math.random()*animationStrength, randNeg()*Math.random()*animationStrength);
					originVector[0] = new Vector3D(vec3D.x, vec3D.y, randNeg()*Math.random()*animationStrength);
					var tweenProxyObject:Object = {x:originVector[0].x, y:originVector[0].y, z:originVector[0].z};
					matrix.recompose(originVector);
					el.setLayoutMatrix3D(matrix, false);

					var tween:GTween = new GTween();
					tween.autoPlay = false;
					tween.target = tweenProxyObject;
					tween.data = {element:el, origin:originVector};
					tween.setValues({x:vec3D.x, y:vec3D.y, z:vec3D.z});
					tween.ease = animationEase;
					tween.duration = animationDuration;
					tween.onChange = function f():void
					{
						var el:ILayoutElement = ILayoutElement(this.data.element);
						var mtx:Matrix3D = el.getLayoutMatrix3D();
						Vector.<Vector3D>(this.data.origin)[0] = new Vector3D(this.target.x, this.target.y, this.target.z);
						mtx.recompose(Vector.<Vector3D>(this.data.origin));
						el.setLayoutMatrix3D(mtx, false);
					}
					tween.paused = false;
				}
				else
				{
					var newVector:Vector.<Vector3D> = matrix.decompose();
					newVector[0] = vec3D;
					matrix.recompose(newVector);
					el.setLayoutMatrix3D(matrix, false);
				}
				elementCounter++;
			}
			else
			{
				UIComponent(el).visible = false;
			}
		}
		catch (error:RangeError)
		{
			//trace(error.message);
		}
	}

	// Remove any remaining elements from the grid
	if (elementCounter < numElements)
	{
		for (var v:int = elementCounter; v < numElements; v++)
		{
			el = target.getElementAt(v);
			UIComponent(el).visible = false;
		}
	}
}

<s:DataGroup
	id="dgThumbnails"
	dataProvider="{this.dataProvider}"
	width="400"
	height="400">
	<s:layout>
		<layouts:ImageLayout
			animate="true"
			animationDuration="1"
			animationEase="Back.easeInOut"
			animationStrength="800"
			depthFactor="-15"
			gridInterval="35"
			contrastThreshold="40"
			id="imageLayout"/>
	</s:layout>
</s:DataGroup>

The animation uses Grant Skinner’s GTween library, which at a small size provides several awesome features and wicked performance. If you find the performance of the app slowing down because of the grid density (anything above 500 elements tends to kill the framerate), you can turn animation off with the checkbox control on the left, which should help when you modify the bitmap source.

Anyhow, that’s it! I hope you’ve enjoyed this tutorial and feel free to ask any questions; I will be more than happy to answer anything I can! Stay tuned for more.

Follow me on twitter here.

IN THE BEGINNING

Alright, so I really have to thank Jay Clegg up front for setting this in motion. Without his existing Peggy and Arduino code, this ditty of a project would not have been possible. Take a look at his article for help with wiring.

I’ve been looking for a good Flex/Flash/LED solution for some time now, but nothing had really caught my attention until I read Jay’s article. The Peggy 2.0 was perfect; tons of LEDs, super fast bus and downright infamous (actually those aren’t Peggys, they are what inspired the Peggys). This isn’t to say creating an array of BlinkM LEDs or some other solution isn’t worth considering, it’s just that the Peggy has the whole package built right in.

So let’s get started! Download the source here.

THE BASICS

The Peggy can render video using the I²C/TWI/SPI interface. Fortunately it’s easy to get an Arduino to output data over TWI/SPI. Better yet, it’s easy to get data to the Arduino from almost any programming language. So the basic setup is this.

Simple enough. The code that Jay Clegg wrote for the Peggy listens over TWI for a packet that contains a 0xDEADBEEF10 header and a series of brightness values defined by bits, which is sent to a frame buffer as fast as Peggily-possible (60Hz). This means I needed to generate a byte array in my programming language of choice, Actionscript 3, and get it to the Arduino, which would send it to the Peggy over TWI.

GOALS

Now that I had my theoretical hardware setup, what was I going to do with it? Render a grid of pixels in Flash (in this case 25×25) and send it over to the Peggy with little to no latency at >30FPS of course! I knew this was possible using Quartz/Objective-C, but I didn’t know if it was possible using Flash.

In the past, the only way I could get flash Flash to communicate with hardware was through a socket connection. That’s how Phidgets do it, that’s how Arduino does it, and that’s how I did it, though it’s not the only way (Kevin Hoyt told me recently to use the Native Process API, more to come). I went with the new UDP Datagram Class in Air 2.0 for its low latency streaming advantages to try and achieve the speed I needed.

I also needed some type of proxy for the Arduino since Flash can’t communicate with it directly. Java has always been my crap-flash-can’t-do-this failsafe and I knew it wouldn’t let me down this round. Writing a UDP server is simple enough, thus I created a UDP data bridge in Java and I was on my way. In the code fragment below, the Outputstream is the serial connection to the Arduino through the RxTx Java library – refer to the source code for the full class.

    /*
    ** This threads listens for any incoming UDP packets and sends them to the Arduino
    */

    public static class SerialWriter implements Runnable
    {
        OutputStream out;
        DatagramSocket datagramSocket;
        Boolean connected = true;

        public SerialWriter (OutputStream out, DatagramSocket datagramSocket)
        {
        	super();
        	this.out = out;
            this.datagramSocket = datagramSocket;
        }

        public void run()
        {
        	// Set connected to false if you want to disconnect, restart the thread to startup again
        	while (connected)
        	{
	            try
	            {
	            	byte[] buffer = new byte[Integer.parseInt(prop.getProperty("udp.buffer.size"))];
	            	DatagramPacket incomingPacket = new DatagramPacket(buffer, buffer.length, InetAddress.getLocalHost(), Integer.parseInt(prop.getProperty("outgoing.udp.port")));
	                datagramSocket.receive(incomingPacket);

	                if (incomingPacket.getLength() > 0)
	                {
	                	this.out.flush();
		                this.out.write(incomingPacket.getData());
	                }
	            }
	            catch ( IOException e )
	            {
	                e.printStackTrace();
	            }
        	}
        	datagramSocket.close();
        }
    }

There are a lot of great resources at the Arduino website regarding Java proxies, as well as many other methods for getting different languages to speak with Arduinos. I highly recommend checking the site out.

AVR PROGRAMMING

Now I was cooking, but I had a slight problem. You see, Jay Clegg is a hardware Jedi master, and he uses an AVR Programmer to program his ATMega328P (the Arduino/Peggy processor). I am simpleton and wanted to use the standard ol’ Arduino USB programming method through the Arduino IDE. Jay wrote his Arduino and Peggy 2.0 code for WinAVR, which is not compatible with the Arduino IDE and looks a lot like C (because it is).

Luckily though, he included some wonderful comments in his source code explaining how to port it over to the Arduino IDE. After following his instructions and adding the loop() and setup() methods, I only needed to change a few global variables (namely PCINT13 and PCINT14 for the pullups) in both code bases and I was good to go!

So here is the sketch for the Arduino, which compiles all nice-like in the Arduino IDE. It’s so easy, even I can do it!™ EDIT: My friends at Evil Mad Science were happy to inform me that they now provide their own Peggy TWI Arduino sketch, which can be found here.

#include <Wire.h>
#define PEGGY_ADDRESS 34
#define TWI_FREQ 300000

void setup()
{
  Serial.begin(115200);

  UBRR0H = 0;
  UBRR0L = 16; // manually set 115200
  UCSR0A = (1<<U2X0);
  Wire.begin();

  PORTC |=  (1<<PCINT13) | (1<<PCINT12); // enable pullups

  TWSR &= ~(1<<TWPS0);
  TWSR &= ~(1<<TWPS1);
  TWBR = ((F_CPU / TWI_FREQ) - 16) / 2;
}

void loop()
{
  uint8_t count = Serial.available();

  if (count > 0)
  {
    if (count > 16) count = 16;

    Wire.beginTransmission(PEGGY_ADDRESS);
    while (count-- > 0 )
    {
      uint8_t c = Serial.read();
      Wire.send(c);
    }
    Wire.endTransmission();
  }
}

The Peggy code is a lot more complicated, so please open it in the Arduino IDE and take a look there (the code has been ported and should work fine). Keep in mind that the Peggy does not have a USB port, so it must be programmed through a USB-TTL cable, though the method is identical to the Arduino.

Alright, so the pieces were coming together. I had the Arduino sending data to the Peggy, and my Java UDP proxy was just begging for some bits. Now it was time for a little Actionscript.

ALL YOUR BASE ARE BELONG TO US

The first smattering of code I wrote was a wrapper around the Datagram class. It needed some type of send(ByteArray) method so that I could render a byte array on a timer event, or on enter frame, or whatever and send it on through to the Java proxy. Thus the following class was created.

		/*
		** Takes a prepared bytearray, adds the proper header and sends it to the UDP Java Proxy
		*/

		public function sendToProxy(ba:ByteArray):void
		{
			var packet:ByteArray = new ByteArray();
			packet.writeBytes(_header, 0, _header.length);

			/*
			** Expecting a ByteArray of 330 bytes, which is 25 rows * 13 Bytes (1 nibble (4 bits) per LED,
			** 16 levels of brightness) plus the header, plus an extra byte due to odd rows
			*/

			packet.writeBytes(ba, 0, ba.length);
			_datagramSocket.send(packet, 0, 0, host, outgoingPort);
			if (exposeByteArray) outputStream = ba;
		}

You’ll notice that the 0xDEADBEEF10 header is appended to every byte array that comes through the send() method. This makes life a lot easier.

On top of having the send() method, I wanted a convenience method that could take a DisplayObject and convert it into a byte array that the Peggy understood. Thus the createStream(DisplayObject) method was born. I could then create a source property on the Proxy and delegate a Timer class to render it out on a set interval (30ms seems to work best). The start() and stop() methods control the Timer. You may also notice that you can expose the raw byte array (exposeByteArray setter) for further processing in the application, such as a visualization component (also included in the source download).

		protected function createStream(component:DisplayObject):void
		{
			if (component.width < 25 || component.height < 25) return;

			var bmd:BitmapData = new BitmapData(component.width, component.height, false, 0x000000);
			var point:Point = new Point(bmd.width/2, bmd.height/2);
			bmd.draw(component, rotateAndFlip(bmd, -90), null, null, null, true); // Positions image on Peggy properly
			bmd.applyFilter(bmd, bmd.rect, new Point(), colorMatrixFilter);
			if (bmd.width > 25 || bmd.height > 25)
			{
				bmd = cropBitmap(bmd, 25, 25);
				bmd = ImageResizer.bilinearIterative(bmd, 25, 25, ResizeMath.METHOD_PAN_AND_SCAN);
			}

			var ba:ByteArray = new ByteArray();
			var average:uint = averageColor(bmd);
			for (var i:int = 0; i < bmd.width; i++)
			{
				var count:int = 0;
				for (var k:int = 0; k < Math.round(bmd.height/2); k++)
				{
					/*
					** We gather two pixels at a time, since a byte equals two pixels.
					** There will always be an extra byte because the rows are odd, which is discarded in the peggy firmware.
					*/

					var pair:int = 0;
					pair = (getBrightness(bmd.getPixel(i, count), average) | getBrightness(bmd.getPixel(i, count+1), average) << 4);
					ba.writeByte(pair);
					count += 2;
				}
			}

			sendToProxy(ba);
		}

And that’s it! Check out the videos below (sorry for the crappy quality, my buddy with the Canon 5D wasn’t available – if you complain enough I will redo them).

EDIT: When I said 60KHz, I really meant to say 60Hz. 60KHz would be crazy fast, sorry for that.

FURTHERMORE… THE NATIVE PROCESS API (AIR 2.0)

After reading up on the Native Process API, it looks like I can completely bypass UDP altogether and send a raw byte array to a Java program over System.in. I’m going to assume this method is better than UDP (hopefully threaded separately as well), as using too many network connections seems to hinder Flash’s performance a bit. This is not something I have done yet, but is definitely something I will explore in the future, so check back for an update!

CONCLUSION

I hope you’ve enjoyed this tasty little hardware experiment. I’d love to see what others come up with using this code. You have no excuses now, go forth and create the best Christmas light show evar! If you have any questions, feel free to leave a comment or email/GTalk me.

Fierce Hardware Integration Source Files

Fierce Integration Slides (stand alone)

Article coming soon, source code link is above.