Flash to Peggy 2.0 – LED Insanity

IN THE BEGINNING

Alright, so I really have to thank Jay Clegg up front for setting this in motion. Without his existing Peggy and Arduino code, this ditty of a project would not have been possible. Take a look at his article for help with wiring.

I’ve been looking for a good Flex/Flash/LED solution for some time now, but nothing had really caught my attention until I read Jay’s article. The Peggy 2.0 was perfect; tons of LEDs, super fast bus and downright infamous (actually those aren’t Peggys, they are what inspired the Peggys). This isn’t to say creating an array of BlinkM LEDs or some other solution isn’t worth considering, it’s just that the Peggy has the whole package built right in.

So let’s get started! Download the source here.


THE BASICS

The Peggy can render video using the I²C/TWI/SPI interface. Fortunately it’s easy to get an Arduino to output data over TWI/SPI. Better yet, it’s easy to get data to the Arduino from almost any programming language. So the basic setup is this.

Basic Hardware Setup for Flash to Peggy

Simple enough. The code that Jay Clegg wrote for the Peggy listens over TWI for a packet that contains a 0xDEADBEEF10 header and a series of brightness values defined by bits, which is sent to a frame buffer as fast as Peggily-possible (60Hz). This means I needed to generate a byte array in my programming language of choice, Actionscript 3, and get it to the Arduino, which would send it to the Peggy over TWI.


GOALS

Now that I had my theoretical hardware setup, what was I going to do with it? Render a grid of pixels in Flash (in this case 25×25) and send it over to the Peggy with little to no latency at >30FPS of course! I knew this was possible using Quartz/Objective-C, but I didn’t know if it was possible using Flash.

In the past, the only way I could get flash Flash to communicate with hardware was through a socket connection. That’s how Phidgets do it, that’s how Arduino does it, and that’s how I did it, though it’s not the only way (Kevin Hoyt told me recently to use the Native Process API, more to come). I went with the new UDP Datagram Class in Air 2.0 for its low latency streaming advantages to try and achieve the speed I needed.

I also needed some type of proxy for the Arduino since Flash can’t communicate with it directly. Java has always been my crap-flash-can’t-do-this failsafe and I knew it wouldn’t let me down this round. Writing a UDP server is simple enough, thus I created a UDP data bridge in Java and I was on my way. In the code fragment below, the Outputstream is the serial connection to the Arduino through the RxTx Java library – refer to the source code for the full class.

    /*
    ** This threads listens for any incoming UDP packets and sends them to the Arduino
    */

    public static class SerialWriter implements Runnable 
    {
        OutputStream out;
        DatagramSocket datagramSocket;
        Boolean connected = true;
        
        public SerialWriter (OutputStream out, DatagramSocket datagramSocket)
        {
        	super();
        	this.out = out;
            this.datagramSocket = datagramSocket;            
        }
        
        public void run()
        {
        	// Set connected to false if you want to disconnect, restart the thread to startup again
        	while (connected)
        	{
	            try
	            {       
	            	byte[] buffer = new byte[Integer.parseInt(prop.getProperty("udp.buffer.size"))];
	            	DatagramPacket incomingPacket = new DatagramPacket(buffer, buffer.length, InetAddress.getLocalHost(), Integer.parseInt(prop.getProperty("outgoing.udp.port")));
	                datagramSocket.receive(incomingPacket);
	                
	                if (incomingPacket.getLength() > 0)
	                {
	                	this.out.flush();
		                this.out.write(incomingPacket.getData());
	                }
	            }
	            catch ( IOException e )
	            {
	                e.printStackTrace();
	            }
        	}
        	datagramSocket.close();
        }
    }

There are a lot of great resources at the Arduino website regarding Java proxies, as well as many other methods for getting different languages to speak with Arduinos. I highly recommend checking the site out.


AVR PROGRAMMING

Now I was cooking, but I had a slight problem. You see, Jay Clegg is a hardware Jedi master, and he uses an AVR Programmer to program his ATMega328P (the Arduino/Peggy processor). I am simpleton and wanted to use the standard ol’ Arduino USB programming method through the Arduino IDE. Jay wrote his Arduino and Peggy 2.0 code for WinAVR, which is not compatible with the Arduino IDE and looks a lot like C (because it is).

Luckily though, he included some wonderful comments in his source code explaining how to port it over to the Arduino IDE. After following his instructions and adding the loop() and setup() methods, I only needed to change a few global variables (namely PCINT13 and PCINT14 for the pullups) in both code bases and I was good to go!

So here is the sketch for the Arduino, which compiles all nice-like in the Arduino IDE. It’s so easy, even I can do it!™ EDIT: My friends at Evil Mad Science were happy to inform me that they now provide their own Peggy TWI Arduino sketch, which can be found here.

#include <Wire.h>
#define PEGGY_ADDRESS 34
#define TWI_FREQ 300000

void setup()
{
  Serial.begin(115200);

  UBRR0H = 0;
  UBRR0L = 16; // manually set 115200
  UCSR0A = (1<<U2X0);
  Wire.begin();

  PORTC |=  (1<<PCINT13) | (1<<PCINT12); // enable pullups

  TWSR &= ~(1<<TWPS0);
  TWSR &= ~(1<<TWPS1);
  TWBR = ((F_CPU / TWI_FREQ) - 16) / 2;
}

void loop()
{
  uint8_t count = Serial.available();
  
  if (count > 0)
  {
    if (count > 16) count = 16;
  
    Wire.beginTransmission(PEGGY_ADDRESS); 
    while (count-- > 0 )
    {
      uint8_t c = Serial.read(); 
      Wire.send(c);
    }
    Wire.endTransmission();
  }
}

The Peggy code is a lot more complicated, so please open it in the Arduino IDE and take a look there (the code has been ported and should work fine). Keep in mind that the Peggy does not have a USB port, so it must be programmed through a USB-TTL cable, though the method is identical to the Arduino.

Alright, so the pieces were coming together. I had the Arduino sending data to the Peggy, and my Java UDP proxy was just begging for some bits. Now it was time for a little Actionscript.


ALL YOUR BASE ARE BELONG TO US

The first smattering of code I wrote was a wrapper around the Datagram class. It needed some type of send(ByteArray) method so that I could render a byte array on a timer event, or on enter frame, or whatever and send it on through to the Java proxy. Thus the following class was created.


		/*
		** Takes a prepared bytearray, adds the proper header and sends it to the UDP Java Proxy
		*/
		
		public function sendToProxy(ba:ByteArray):void
		{
			var packet:ByteArray = new ByteArray();			
			packet.writeBytes(_header, 0, _header.length);
			
			/* 
			** Expecting a ByteArray of 330 bytes, which is 25 rows * 13 Bytes (1 nibble (4 bits) per LED,
			** 16 levels of brightness) plus the header, plus an extra byte due to odd rows
			*/
			
			packet.writeBytes(ba, 0, ba.length);
			_datagramSocket.send(packet, 0, 0, host, outgoingPort);
			if (exposeByteArray) outputStream = ba;
		}

You’ll notice that the 0xDEADBEEF10 header is appended to every byte array that comes through the send() method. This makes life a lot easier.

On top of having the send() method, I wanted a convenience method that could take a DisplayObject and convert it into a byte array that the Peggy understood. Thus the createStream(DisplayObject) method was born. I could then create a source property on the Proxy and delegate a Timer class to render it out on a set interval (30ms seems to work best). The start() and stop() methods control the Timer. You may also notice that you can expose the raw byte array (exposeByteArray setter) for further processing in the application, such as a visualization component (also included in the source download).

		protected function createStream(component:DisplayObject):void
		{
			if (component.width < 25 || component.height < 25) return;

			var bmd:BitmapData = new BitmapData(component.width, component.height, false, 0x000000);
			var point:Point = new Point(bmd.width/2, bmd.height/2); 
			bmd.draw(component, rotateAndFlip(bmd, -90), null, null, null, true); // Positions image on Peggy properly
			bmd.applyFilter(bmd, bmd.rect, new Point(), colorMatrixFilter);
			if (bmd.width > 25 || bmd.height > 25)
			{
				bmd = cropBitmap(bmd, 25, 25);
				bmd = ImageResizer.bilinearIterative(bmd, 25, 25, ResizeMath.METHOD_PAN_AND_SCAN);
			}
			
			var ba:ByteArray = new ByteArray();
			var average:uint = averageColor(bmd);
			for (var i:int = 0; i < bmd.width; i++)
			{
				var count:int = 0;
				for (var k:int = 0; k < Math.round(bmd.height/2); k++)
				{
					/*
					** We gather two pixels at a time, since a byte equals two pixels.
					** There will always be an extra byte because the rows are odd, which is discarded in the peggy firmware.
					*/
					
					var pair:int = 0;
					pair = (getBrightness(bmd.getPixel(i, count), average) | getBrightness(bmd.getPixel(i, count+1), average) << 4);
					ba.writeByte(pair);
					count += 2;
				}
			}
			
			sendToProxy(ba);
		}

And that’s it! Check out the videos below (sorry for the crappy quality, my buddy with the Canon 5D wasn’t available – if you complain enough I will redo them).


EDIT: When I said 60KHz, I really meant to say 60Hz. 60KHz would be crazy fast, sorry for that. :)


FURTHERMORE… THE NATIVE PROCESS API (AIR 2.0)

After reading up on the Native Process API, it looks like I can completely bypass UDP altogether and send a raw byte array to a Java program over System.in. I’m going to assume this method is better than UDP (hopefully threaded separately as well), as using too many network connections seems to hinder Flash’s performance a bit. This is not something I have done yet, but is definitely something I will explore in the future, so check back for an update!


CONCLUSION

I hope you’ve enjoyed this tasty little hardware experiment. I’d love to see what others come up with using this code. You have no excuses now, go forth and create the best Christmas light show evar! If you have any questions, feel free to leave a comment or email/GTalk me.

  • Pingback: uberVU - social comments

  • vitaLee

    nice post. :)
    subscribed to your blog.

  • admin

    Thanks! I hope to have some more cool stuff up soon.

  • Pingback: Tweets that mention The good times keep coming: general musings of leonard -- Topsy.com

  • http://sites.google.com/site/chrelad chrelad

    This is awesome! This has moved to the front of my “Cool stuff to try” list.

    Thanks for this and keep these awesome posts coming :)

    Chrelad

  • http://www.cna-trainingclass.com/ cna training

    Terrific work! This is the type of information that should be shared around the web. Shame on the search engines for not positioning this post higher!

  • Daniel

    Hello!
    I've downloaded all the files but can't set up the project right in eclipse. Any chance you could write/record a video walkthrough ? I need some kind of tutorial because I'm not used to the clipse(and flex) environment. Thank you in adv,
    Daniel

  • leonardsouza

    @daniel

    Just unzip the Flex project, and in Flash Builder go to File -> Import -> Point to the directory of the project, and click next. The name of the project should show up with a checkbox next to it, click finish. Here is a link to how to import a Java project, but the principles are the same: http://agile.csc.ncsu.edu/SEMaterials/tutorials… – I hope that helps!

  • Daniel

    Still having problems with it, getting “The project cannot be built until build path errors are resolved” and some other stuff on eclipse/flex. What to do ? :(

  • leonardsouza

    Is this the flex or the java project?

  • tilt

    i am also appearing to get the same problem… this is the java project

  • leonardsouza

    which version of eclipse are you using? and what is the exact stack trace?

  • TILT

    I'm running the latest version of eclipse that i just got on the website.
    Well at the beginning i had “build path errors”, such as

    Unbound classpath container: 'JRE System Library [JVM 1.6]' in project 'arduino_serial_proxy'

    So i did a quick fix, and selected the JRE 1.6 environment.
    Then i get the errors relating to CommPort, SerialPort and how i can't import the gnu.io libraries.

    So i tried to add the RXTXComm.jar library to the build path when i try to do a quick manual fix again….
    No problems, until i try to compile and run and get this:

    java.lang.UnsatisfiedLinkError: no rxtxSerial in java.library.path thrown while loading gnu.io.RXTXCommDriver
    Exception in thread “main” java.lang.UnsatisfiedLinkError: no rxtxSerial in java.library.path
    at java.lang.ClassLoader.loadLibrary(Unknown Source)
    at java.lang.Runtime.loadLibrary0(Unknown Source)
    at java.lang.System.loadLibrary(Unknown Source)
    at gnu.io.CommPortIdentifier.<clinit>(CommPortIdentifier.java:83)
    at com.whiterabbit.arduino.SerialProxy.connect(SerialProxy.java:62)
    at com.whiterabbit.arduino.SerialProxy.main(SerialProxy.java:215)
    Did i do anything wrong during this process??

    I think it'll much easier if you could guide us in a step by step manner in how to get all of this working…
    I haven't had any experience using eclipse or the Flash builder.

  • leonardsouza

    You did a great job and just outlined the steps above needed to make it work! As for that error, you need to go into the Arduino IDE and find your arduino device ID (Tools -> Serial Port -> find the ID). and edit the config file inside the project (filename: whiterabbit.properties). Replace the existing device ID with yours. That should remove that error! Let me know if you need anymore help.

  • TILT

    Okay, i changed the whiterabbit.properties file such that port.name=COM3, because my Arduino is on COM3.
    Also, you forgot to mention how i needed the rxtxSerial.dll file in my jre directory.
    Everything seems to work fine for the java part of the process.
    However, i am now facing the problem with flash builder when i import the project whereby i get,

    Unknown Flex SDK: “Flex 4.0 / Air 2.0″ error

    Therefore, i assumed the project Flex compiler and build path was incorrect again, so i chose the defualt Flex 4.0.

    This prompted me with a new error:

    1046: Type was not found or was not a compile-time constant: DatagramSocketDataEvent. PeggyProxy.as /fierce_led_matrix/src/com/whiterabbit/arduino/peggy line 134

    Any hints on what i should be doing or have done wrong?
    Thanks for all the help!

  • leonardsouza

    You have done nothing wrong, you just don't have the Air 2.0 Beta SDK installed. Please refer to the following link for how to overlay the beta SDK into the Flex 4 SDK: http://labs.adobe.com/wiki/index.php/AIR_2:Rele

    The “Flex 4 / Air 2.0″ SDK my project was referring to was created by following the steps in the above link. The Datagram class isn't available anywhere but the Air 2.0 SDK. Let me know if that works.

  • TILT

    Alright, managed to overlay the Air 2.0 with Flex 4 SDK, so that error disappeared.
    But now i get a NEW error:

    1046: Type was not found or was not a compile-time constant: VideoElement. Main.mxml /fierce_led_matrix/src line 43 Flex Problem

    1180: Call to a possibly undefined method VideoElement. Main.mxml /fierce_led_matrix/src line 67 Flex Problem

    When i run it, the application opens, but its not showing on my Peggy, only because i think these two errors are probably why.
    Any more hints? I have a feeling i'm really close to getting this work :D
    Thanks again, and sorry if this is such a hassle for you!

  • leonardsouza

    Ah yes, I built that project against an older version of the Flex 4 SDK (contained a reference to VideoElement, which has changed to VideoDisplay). I updated the project inside fierce_files.zip to work with the new SDK, please download it and let me know if it works for you. Here is a link directly to the FXP: http://bit.ly/aQyEnl (right-click and click save link as – make sure it's named *.fxp).

  • TILT

    YES!!! Finally got it to work now!!!
    The only thing that didn't work for me was the webcam option.
    i get:
    TypeError: Error #1009: Cannot access a property or method of a null object reference.
    at Main/dropdownChangeHandler()[C:UsersAlexAdobe Flash Builder 4fierce_led_matrix_1srcMain.mxml:116]
    at Main/__peggyDropDown_change()[C:UsersAlexAdobe Flash Builder 4fierce_led_matrix_1srcMain.mxml:235]
    at flash.events::EventDispatcher/dispatchEventFunction()
    at flash.events::EventDispatcher/dispatchEvent()
    at mx.core::UIComponent/dispatchEvent()[E:dev4.0.0frameworksprojectsframeworksrcmxcoreUIComponent.as:12266]
    at spark.components::List/commitSelection()[E:dev4.0.0frameworksprojectssparksrcsparkcomponentsList.as:1047]
    at spark.components.supportClasses::DropDownListBase/commitSelection()[E:dev4.0.0frameworksprojectssparksrcsparkcomponentssupportClassesDropDownListBase.as:561]
    at spark.components.supportClasses::ListBase/commitProperties()[E:dev4.0.0frameworksprojectssparksrcsparkcomponentssupportClassesListBase.as:829]
    at spark.components::List/commitProperties()[E:dev4.0.0frameworksprojectssparksrcsparkcomponentsList.as:905]
    at spark.components.supportClasses::DropDownListBase/commitProperties()[E:dev4.0.0frameworksprojectssparksrcsparkcomponentssupportClassesDropDownListBase.as:504]
    at spark.components::DropDownList/commitProperties()[E:dev4.0.0frameworksprojectssparksrcsparkcomponentsDropDownList.as:267]
    at mx.core::UIComponent/validateProperties()[E:dev4.0.0frameworksprojectsframeworksrcmxcoreUIComponent.as:7772]
    at spark.components::List/item_mouseDownHandler()[E:dev4.0.0frameworksprojectssparksrcsparkcomponentsList.as:1541]
    at spark.components.supportClasses::DropDownListBase/item_mouseDownHandler()[E:dev4.0.0frameworksprojectssparksrcsparkcomponentssupportClassesDropDownListBase.as:728]

    Thanks for all of your help!

  • leonardsouza

    Ya, I didn't entirely test the app under the new SDK. :) I think if you try the webcam twice it will work fine, otherwise you may have to go into the code yourself and figure out where the null is coming from.

  • Daniel

    Hello again, still can't get it to work, I need some help..I'm getting this error:

    at gnu.io.CommPortIdentifier.getPortIdentifier(CommPortIdentifier.java:218)
    at com.whiterabbit.arduino.SerialProxy.connect(SerialProxy.java:62)
    at com.whiterabbit.arduino.SerialProxy.main(SerialProxy.java:215)

    What to do ? :(

    Thanks in adv.

  • leonardsouza

    Hi there! Tilt was having the same problem, please view his thread regarding that error. You will need to edit the whiterabbit.properties file. Let me know if you still have some problems.

  • Pingback: Pixels, passion + purpose. » Blog Archive » Flash to Peggy 2.0 – LED Insanity

  • Daniel

    Is it possible to use this code for 2 Peggys ? One Peggy is not enough for text scrolling :P
    What about the wiring ?
    Thanks

  • leonardsouza

    Unfortunately the TWI interface is managed via the Wire Arduino library, which utilizes two specific outs on the Arduino. I'm not sure if any of the other ports on the Arduino can be used for TWI, but if so, you'd probably have to look at the Wire library code and rewrite it to output to multiple wires (you may need a different type of Arduino, not sure). Otherwise, you can always use two Arduinos and synchronize them. :) Let me know if you figure it out!

  • http://www.louisvuittonhandbagmall.com/ Louis

    Awesome,
    these sketches made me to visit the awesome design processes..
    Thanks Sneh for great collection.

  • Msncyp

    HI!
    I am using FB4 and just try your new version of FXP (http://bit.ly/aQyEnl),
    I choose AIR 3.5 SDK and i got some error on it.

    Description Resource Path Location Type
    Could not resolve to a component implementation. Main.mxml /fierce_led_matrix/src line 12 Flex Problem

    anything i can do??

  • Anonymous

    Are you sure it’s 3.5? I think it might be looking for 2.5, which means you’d need to updated the namespace in the Main-app.xml to “http://ns.adobe.com/air/application/2.6″ (or 2.5). Let me know if that works.

  • Msncyp

    Sorry i really can’t do it.
    Maybe i show you the my setting, hope it help to illustrate the problem.

    http://soimperfect.com/problem_encounter_1.JPG
    http://soimperfect.com/problem_encounter_2.JPG

    Thanks!!

  • Anonymous

    The latest version of AIR is 2.6 (it’s not the same as the Flex SDK). Try changing the SDK to Flex 4 and the namespace to

  • Msncyp

    Hi,
    I finally can work with Peggy with Flex. Thanks a lot!!
    But i would like to ask one more question, how can i display images using that Flex project?

    Thanks a lot!!

  • Anonymous

    Anything you that is added to peggyGroup in Main.mxml will be drawn to the Peggy. So, just add an s:Image to the group and you are good to go. :)

  • Msncyp

    Thanks!!! :)