Flash to Peggy 2.0 – LED Insanity

IN THE BEGINNING

Alright, so I really have to thank Jay Clegg up front for setting this in motion. Without his existing Peggy and Arduino code, this ditty of a project would not have been possible. Take a look at his article for help with wiring.

I’ve been looking for a good Flex/Flash/LED solution for some time now, but nothing had really caught my attention until I read Jay’s article. The Peggy 2.0 was perfect; tons of LEDs, super fast bus and downright infamous (actually those aren’t Peggys, they are what inspired the Peggys). This isn’t to say creating an array of BlinkM LEDs or some other solution isn’t worth considering, it’s just that the Peggy has the whole package built right in.

So let’s get started! Download the source here.

THE BASICS

The Peggy can render video using the I²C/TWI/SPI interface. Fortunately it’s easy to get an Arduino to output data over TWI/SPI. Better yet, it’s easy to get data to the Arduino from almost any programming language. So the basic setup is this.

Simple enough. The code that Jay Clegg wrote for the Peggy listens over TWI for a packet that contains a 0xDEADBEEF10 header and a series of brightness values defined by bits, which is sent to a frame buffer as fast as Peggily-possible (60Hz). This means I needed to generate a byte array in my programming language of choice, Actionscript 3, and get it to the Arduino, which would send it to the Peggy over TWI.

GOALS

Now that I had my theoretical hardware setup, what was I going to do with it? Render a grid of pixels in Flash (in this case 25×25) and send it over to the Peggy with little to no latency at >30FPS of course! I knew this was possible using Quartz/Objective-C, but I didn’t know if it was possible using Flash.

In the past, the only way I could get flash Flash to communicate with hardware was through a socket connection. That’s how Phidgets do it, that’s how Arduino does it, and that’s how I did it, though it’s not the only way (Kevin Hoyt told me recently to use the Native Process API, more to come). I went with the new UDP Datagram Class in Air 2.0 for its low latency streaming advantages to try and achieve the speed I needed.

I also needed some type of proxy for the Arduino since Flash can’t communicate with it directly. Java has always been my crap-flash-can’t-do-this failsafe and I knew it wouldn’t let me down this round. Writing a UDP server is simple enough, thus I created a UDP data bridge in Java and I was on my way. In the code fragment below, the Outputstream is the serial connection to the Arduino through the RxTx Java library – refer to the source code for the full class.

    /*
    ** This threads listens for any incoming UDP packets and sends them to the Arduino
    */

    public static class SerialWriter implements Runnable
    {
        OutputStream out;
        DatagramSocket datagramSocket;
        Boolean connected = true;

        public SerialWriter (OutputStream out, DatagramSocket datagramSocket)
        {
        	super();
        	this.out = out;
            this.datagramSocket = datagramSocket;
        }

        public void run()
        {
        	// Set connected to false if you want to disconnect, restart the thread to startup again
        	while (connected)
        	{
	            try
	            {
	            	byte[] buffer = new byte[Integer.parseInt(prop.getProperty("udp.buffer.size"))];
	            	DatagramPacket incomingPacket = new DatagramPacket(buffer, buffer.length, InetAddress.getLocalHost(), Integer.parseInt(prop.getProperty("outgoing.udp.port")));
	                datagramSocket.receive(incomingPacket);

	                if (incomingPacket.getLength() > 0)
	                {
	                	this.out.flush();
		                this.out.write(incomingPacket.getData());
	                }
	            }
	            catch ( IOException e )
	            {
	                e.printStackTrace();
	            }
        	}
        	datagramSocket.close();
        }
    }

There are a lot of great resources at the Arduino website regarding Java proxies, as well as many other methods for getting different languages to speak with Arduinos. I highly recommend checking the site out.

AVR PROGRAMMING

Now I was cooking, but I had a slight problem. You see, Jay Clegg is a hardware Jedi master, and he uses an AVR Programmer to program his ATMega328P (the Arduino/Peggy processor). I am simpleton and wanted to use the standard ol’ Arduino USB programming method through the Arduino IDE. Jay wrote his Arduino and Peggy 2.0 code for WinAVR, which is not compatible with the Arduino IDE and looks a lot like C (because it is).

Luckily though, he included some wonderful comments in his source code explaining how to port it over to the Arduino IDE. After following his instructions and adding the loop() and setup() methods, I only needed to change a few global variables (namely PCINT13 and PCINT14 for the pullups) in both code bases and I was good to go!

So here is the sketch for the Arduino, which compiles all nice-like in the Arduino IDE. It’s so easy, even I can do it!™ EDIT: My friends at Evil Mad Science were happy to inform me that they now provide their own Peggy TWI Arduino sketch, which can be found here.

#include <Wire.h>
#define PEGGY_ADDRESS 34
#define TWI_FREQ 300000

void setup()
{
  Serial.begin(115200);

  UBRR0H = 0;
  UBRR0L = 16; // manually set 115200
  UCSR0A = (1<<U2X0);
  Wire.begin();

  PORTC |=  (1<<PCINT13) | (1<<PCINT12); // enable pullups

  TWSR &= ~(1<<TWPS0);
  TWSR &= ~(1<<TWPS1);
  TWBR = ((F_CPU / TWI_FREQ) - 16) / 2;
}

void loop()
{
  uint8_t count = Serial.available();

  if (count > 0)
  {
    if (count > 16) count = 16;

    Wire.beginTransmission(PEGGY_ADDRESS);
    while (count-- > 0 )
    {
      uint8_t c = Serial.read();
      Wire.send(c);
    }
    Wire.endTransmission();
  }
}

The Peggy code is a lot more complicated, so please open it in the Arduino IDE and take a look there (the code has been ported and should work fine). Keep in mind that the Peggy does not have a USB port, so it must be programmed through a USB-TTL cable, though the method is identical to the Arduino.

Alright, so the pieces were coming together. I had the Arduino sending data to the Peggy, and my Java UDP proxy was just begging for some bits. Now it was time for a little Actionscript.

ALL YOUR BASE ARE BELONG TO US

The first smattering of code I wrote was a wrapper around the Datagram class. It needed some type of send(ByteArray) method so that I could render a byte array on a timer event, or on enter frame, or whatever and send it on through to the Java proxy. Thus the following class was created.

		/*
		** Takes a prepared bytearray, adds the proper header and sends it to the UDP Java Proxy
		*/

		public function sendToProxy(ba:ByteArray):void
		{
			var packet:ByteArray = new ByteArray();
			packet.writeBytes(_header, 0, _header.length);

			/*
			** Expecting a ByteArray of 330 bytes, which is 25 rows * 13 Bytes (1 nibble (4 bits) per LED,
			** 16 levels of brightness) plus the header, plus an extra byte due to odd rows
			*/

			packet.writeBytes(ba, 0, ba.length);
			_datagramSocket.send(packet, 0, 0, host, outgoingPort);
			if (exposeByteArray) outputStream = ba;
		}

You’ll notice that the 0xDEADBEEF10 header is appended to every byte array that comes through the send() method. This makes life a lot easier.

On top of having the send() method, I wanted a convenience method that could take a DisplayObject and convert it into a byte array that the Peggy understood. Thus the createStream(DisplayObject) method was born. I could then create a source property on the Proxy and delegate a Timer class to render it out on a set interval (30ms seems to work best). The start() and stop() methods control the Timer. You may also notice that you can expose the raw byte array (exposeByteArray setter) for further processing in the application, such as a visualization component (also included in the source download).

		protected function createStream(component:DisplayObject):void
		{
			if (component.width < 25 || component.height < 25) return;

			var bmd:BitmapData = new BitmapData(component.width, component.height, false, 0x000000);
			var point:Point = new Point(bmd.width/2, bmd.height/2);
			bmd.draw(component, rotateAndFlip(bmd, -90), null, null, null, true); // Positions image on Peggy properly
			bmd.applyFilter(bmd, bmd.rect, new Point(), colorMatrixFilter);
			if (bmd.width > 25 || bmd.height > 25)
			{
				bmd = cropBitmap(bmd, 25, 25);
				bmd = ImageResizer.bilinearIterative(bmd, 25, 25, ResizeMath.METHOD_PAN_AND_SCAN);
			}

			var ba:ByteArray = new ByteArray();
			var average:uint = averageColor(bmd);
			for (var i:int = 0; i < bmd.width; i++)
			{
				var count:int = 0;
				for (var k:int = 0; k < Math.round(bmd.height/2); k++)
				{
					/*
					** We gather two pixels at a time, since a byte equals two pixels.
					** There will always be an extra byte because the rows are odd, which is discarded in the peggy firmware.
					*/

					var pair:int = 0;
					pair = (getBrightness(bmd.getPixel(i, count), average) | getBrightness(bmd.getPixel(i, count+1), average) << 4);
					ba.writeByte(pair);
					count += 2;
				}
			}

			sendToProxy(ba);
		}

And that’s it! Check out the videos below (sorry for the crappy quality, my buddy with the Canon 5D wasn’t available – if you complain enough I will redo them).

EDIT: When I said 60KHz, I really meant to say 60Hz. 60KHz would be crazy fast, sorry for that.

FURTHERMORE… THE NATIVE PROCESS API (AIR 2.0)

After reading up on the Native Process API, it looks like I can completely bypass UDP altogether and send a raw byte array to a Java program over System.in. I’m going to assume this method is better than UDP (hopefully threaded separately as well), as using too many network connections seems to hinder Flash’s performance a bit. This is not something I have done yet, but is definitely something I will explore in the future, so check back for an update!

CONCLUSION

I hope you’ve enjoyed this tasty little hardware experiment. I’d love to see what others come up with using this code. You have no excuses now, go forth and create the best Christmas light show evar! If you have any questions, feel free to leave a comment or email/GTalk me.