Jul 26 2010

Making PDFs and SWFs talk to each other

Lately I’ve been thinking on what to blog and either haven’t found anything particular to talk about or just don’t have the will to do it. I have a bunch of unpublished posts on hold due to lack of time to finish the code samples and what not, so I decided to post a bit on what I have my hands on as of today.  Perhaps later I’ll try to just finish what’s left undone, perhaps not.

Anyway, these days I’ve been working on a crazy PDF – Flash integration and have been getting my hands dirty on Acrobat Javascript and figuring out what would be the best way to make a SWF file talk to a PDF document and vice verse — I have to accept though that I was not even aware that a PDF document could be scripted, WTF!, and further on do all sorts of crazy sh!t like accessing databases, consume web services, etc.

Given that the integration had to happen inside the web browser, using Adobe Air was not an option, so the first item on the TODO list was to be able to “render” a PDF document within a flash movie. I quoted render because the flash player does not support it so the approach (or hack if you will) consisted on placing the PDF object on top of the SWF giving the fake impression that the PDF is contained within the SWF application. Once I was able to place my PDF doc on top of my SWF, I had to figure out how to make both of them talk to each other. To my surprise, it turned out to be very simple.

To keep things simple, the sole requirement is to make our SWF movie to respond-or-act depending on the interaction that our user has on a PDF form. Let see what it takes to accomplish this task.

(Click here to see the example, tested in Firefox and Chrome, and the code, right-click view source enabled. Obviously it is required to have a Adobe Reader installed)

First, we’ll create our flash application. I’m using Flex 3 by the way, but you are completely free to do it in whatever version you like ;) Basically this “app” will have a button to get our PDF document, and the necessary visual elements to display whenever the user does something within the interactive PDF form. And a special container that will act as our PDF placeholder and also take care of all of its position and resizing actions.

The most important aspect of our Flex application is the canvas control, as it will serve as our PDF document placeholder. Obviously the PDF document won’t be loaded within the canvas control, because is not possible, but what it’ll do instead is determine the position and size of our PDF object that will get loaded and displayed within our HTML page. This PDF placeholder, which I repeat is a Canvas, has the following code that is used to load the PDF object (by the way is a total rip-off, plus a couple of other things I needed, of the IFrame component that can be found over here, thanks guys):

public function set source(source: String): void
        {
            if (source)
            {
                if (! ExternalInterface.available)
                {
                    throw new Error("ExternalInterface is not available in this container. Internet Explorer ActiveX, Firefox, Mozilla 1.7.5 and greater, or other browsers that support NPRuntime are required.");
                }
                __source = source;
                ExternalInterface.call("loadPDFObject", source);
                movePDFObject();
            }
        }

, basically what the code on the PDFContainer class does is invoke some javascript functions, through ExternalInterface, to load the PDF document and place it within a previously define HTML DIV. Also, has some other methods to handle resizing and positioning in case we want to re-size or reposition our PDF placeholder canvas within our flex application. For this particular example I won’t make use of any reisizing and reposition stuff, other than the initial.

Ok, I will move on to the PDF part now, and we’ll get back to our Flex app later. As I mentioned at the beginning of this post, you can do all kinds of things within a PDF document – you can add some Javascript to it in order for it to communicate with databases, web services, etc. For our example we are only going to add the necessary stuff to move around data between our PDF document and our Flex application. Acrobat Javascript is basically a framework built on top javascript, so you pretty much can do all the things you’d normally do with javascript, plus a lot of other cool stuff related to the PDF themselves.

It is worth mentioning that all javascript within a PDF happens within its contextt, it is isolated from the external container in case that the PDF is embedded within an HTML page. That being said some of the JS functions that you’d normally use on an HTML container don’t have the same effect within the PDF document, for example, to display an alert on a HTML page you would do:

alert("My Alert");

If you script that on your PDF document nothing will happen, no alert will pop out. In order to display an alert within the PDF document you’ll do instead

app.alert("My Alert");

The “app” object bascailly represents the Acrobat application and provides you access to a bunch functions and routines that are specific to Acrobat. On the other hand there is the “Document” object, which is the most important as it enables the communication between the PDF document itself, the one that is in the viewer, and the Javascript interpreter. Through that object, which usually you refer to it using “this”, you can use some handy methods and properties for accessing the PDF document. For example, let say you have a form that contains a text field named “MySuperCoolTxtField” (pretty lame name I know), you could get its value like:

this.getField("MySuperCollTxtField").value

, it sorts of reminds me to the getElementById method available in JS. Doesn’t it ? Anyway, this is not intended to be an Acrobat JS tutorial nor I do have the expertise to elaborate on one, so if you are interested on going furhter with Arobat JS here are some helpful guides.

Moving on to you initial purpose … where was I ?

Ok, now we are going to create a simple PDF form (here you can take a look at it) to make our little test. Well I actually have it ready, so I’ll just mention pretty much the gist. Here is the final PDF form. It is worth mentioning that you’ll need Acrobat Pro in order to be able to create interactive forms and add JS to your documents and stuff. Basically what the form will do is to trigger events whenever one of its controls got/loses focus, which will be hopefully catched by our HTML JS.

If you take a look at the form you’ll see that we have 4 controls, 3 textfields and one button. We have NameTxt, LocationTxt, and EmailTxt, and the SaveBtn. Each of the text controls have two actions attached. An action represents an function that you would like tog get executed as a response of an event triggered by the control. For example for this text controls Acrobat allows you to add actions for the events MouseUp, MouseDown, MouseExit, OnBlur, OnFocus, and MouseEnter. For this example I just added actions for the OnBlur and OnFocus events of all of the three text controls. See screenshot below.

Actions

On the OnFocus action we have this.sendOnFocusMessage("NameTxt");, and on the OnBlur action we have this.sendOnBlurMessage("NameTxt"); . If we click on Edit with a particular action selected an editor will pop out to display the current code associated to that action, if any. We have the same function for all of the other controls with the difference that we send its correspoing name as the function argument (“LocationTxt” and “EmailTxt” respectively).

If you recall, “this” refers to the Document it self, so we are calling a function that exists within the document scope. You can add as many code to your document as you wish, if you happen to have your Acrobat ready you’ll need to go to Main Menu -> Advanced -> Document Processing -> Document Javascripts, which will open a window that will list all of the code/functions associated with your Document.

This is the code that we have on those functions,

function sendOnBlurMessage(fieldName)
{
    if(!this.hostContainer)
        app.alert("No hostContainer object found");
    else
    {
        try
        {
            var fieldValue = this.getField(fieldName).value;
            if( fieldValue.length == 0 ) fieldValue = " "; 
 
            this.hostContainer.postMessage(["OnBlur", fieldName, fieldValue]);
        }
        catch(e)
        {
            //Display some error
        }
    }
}
 
function sendOnFocusMessage(fieldName)
{
    if(!this.hostContainer)
        app.alert("No hostContainer object found");
    else
    {
        try
        {
            this.hostContainer.postMessage(["OnFocus", fieldName]);
        }
        catch(e)
        {
            //Display some error, this alert will be displayed within the PDF document
        }
    }
}

Ok, before I go any further let me point out a couple of things … If you look at the above code, they pretty much looks normal, however there is a dude called hostContainer. WTF? Ok, hostContainer is an object that your document has access to in order to communicate with its host container, DUH!!!! that sounded stupid, for example an HTML page; hostContainer will be null if you open your document on your PDF viewer application, otherwise, when opened inside an HTML, this object will hold a reference to it. Look at hostContainer as something along the lines of ExternalInterface. The hostContainer basically uses the postMessage function to send messages, and also can have event handlers registered in order to respond to specific actions. Notice that the argument that postMessage expects, which is mandatory, is an array of strings.

As to what the functions do, well, basically whenever a control looses focus they send a message to tell the container was it, and when they get focus also pass the data entered by the user. I guess we have everything we need on our PDF document, so let’s move on to our actual HTML JS. That was simple!

In our HTML javascript we need to do a couple of things – first register message handlers to our PDF object, and create our actual handlers. So, we ended up with something like the following:

function registerPDFMessageHandlers()
{
	try
	{
		var PDFObject = document.getElementById("PDFObjectID");
		PDFObject.messageHandler = {
				onMessage:onMessageHandler,
				onError:onErrorHandler
		};
	}
	catch(e)
	{
		alert("Error: " + e);
	}
}
 
function onMessageHandler(params)
{
	// params - Array
	// 0: Action triggered
	// 1: FieldName
	// 2: FieldValue
	var action = params[0];
	var fieldName = params[1];
	var fieldValue = params[2];
 
	var swf = getMovie("PDFSample");
 
	if( swf )
	{
		switch( action )
		{
			case Action.onFocus:
				swf.sendOnFocus(fieldName);
			break;
 
			case Action.onBlur:
				swf.sendOnBlur(fieldName, trim(fieldValue));
			break;
		}
	}
	else
	{
		alert( "SWF Object not found" );
	}
}
 
function onErrorHandler(error, message)
{
	alert("Error: " + error);
}

The registerPDFMessageHandlers register the proper message handlers on our PDF object, which means that whenever the postMessage method gets invoked (insider our PDF object) the event handlers will get called. I forgot to mention earlier, but our hostContainer acts on 3 differente evets, onMessage, invoked at a successfully postMessage action, onError and onDisclose. You can take a look at the documentation for further information. registerPDFMessageHandlers is called right after our PDF object is loaded, remeber that the loadPDFObject function is called by our Flex application through ExternalInterface.

The function defined as our onMessage event handler, onMessageHandler, is the one that handles the message data that is passed from our PDF document. What it does is very simple stupid, it basically see whether is recieving an OnBlur or OnFocus action and calls the appropiate SWF functions, previously discussed, that have been exposed by ExternalInterface. It is pretty much self explanatory all of it.

Now, going back to our Flex application…. man! this has been a long post! if you are still reading all of this crap I applaud you, seriously!!! you are the man!!

The last part that I want to go over briefly is the code that handles the javascript calls made after our HTML container recieves PDF news, thus make in it look as if our Flex application is instantly and directly communicating with our PDF document. We have two methods exposed by ExternalInteface, one to handle OnFocus message, and the other one to handle OnBlur messages. We could have on sole method to handle all of the messaging however just for the sake of this example I decided two have one for each of the events happening within our PDF form.

private function _onFocusSendCommand(fieldName:Object):void
{
	if( this.alertWrapper != null )
	{
		this.floatingAlertSpanshot = new UIComponent();
 
		snapshotData = new BitmapData(this.floatingAlertSprite.width, this.floatingAlertSprite.height, true);
		snapshotData.draw(this.floatingAlertSprite, new Matrix());
		snapshotBmp = new Bitmap(snapshotData);
 
		this.floatingAlertSpanshot.x = this.alertWrapper.x;
		this.floatingAlertSpanshot.y = this.alertWrapper.y;
		this.floatingAlertSpanshot.addChild(snapshotBmp);
 
		this.addChild(this.floatingAlertSpanshot);
		this.alertWrapper.alpha = 0;
		this.alertWrapper.removeChild(this.floatingAlertSprite);
		this.floatingAlertSprite = null;
 
		var t:GTween = new GTween(this.floatingAlertSpanshot, 0.2, {alpha:0});
	}
	else
	{
		this.alertWrapper = new UIComponent();
		this.alertWrapper.alpha = 0;
		this.addChild(this.alertWrapper);
	}
 
	var alertMsg:String;
	var alertPos:Point = new Point();
	alertPos.x = this.pdfContainer.x + this.pdfContainer.width + 20;
 
	switch( fieldName )
	{
		case Constants.PDF_NAME_FIELD:
			alertMsg = Constants.ENTER_NAME_MESAGE;
			alertPos.y = this.pdfContainer.y + 190;
		break;
 
		case Constants.PDF_LOCATION_FIELD:
			alertMsg = Constants.ENTER_LOCATION_MESSAGE;
			alertPos.y = this.pdfContainer.y + 230;
		break;
 
		case Constants.PDF_EMAIL_FIELD:
			alertMsg = Constants.ENTER_EMAIL_MESSAGE;
			alertPos.y = this.pdfContainer.y + 270;
		break;
	}
 
	this.floatingAlertSprite = new FloatingAlert(alertMsg, "Important Tip");
 
	alertWrapper.addChild(this.floatingAlertSprite);
	alertWrapper.x = alertPos.x;
	alertWrapper.y = alertPos.y;
 
	var gt:GTween = new GTween(alertWrapper, 0.2, {alpha:1});
}
 
private function _onBlurSendCommand(fieldName:Object, fieldValue:Object):void
{
	var fieldValueStr:String = fieldValue as String;
	this[fieldName] = (fieldValueStr.length != 0 && fieldValueStr != "") ? fieldValueStr : null;
	var infoStr:String = "";
 
	infoStr += (this.NameTxt != null) ? "Your name is '" + this.NameTxt + "'. " : "";
	infoStr += ( this.LocationTxt != null ) ? "You are located at '" + this.LocationTxt + "'. " : "";
	infoStr += ( this.EmailTxt != null ) ? "Ohh, and BTW, this email address '" + this.EmailTxt + "' will get lots of spam from me." : "";
 
	if( infoStr == "" ) infoStr = "Please fill out the form on the left";
 
	this.dataBoard.message = infoStr;
}
 
private function _loadPDF(e:Event):void
{
	this.loadBtn.enabled = false;
	this.pdfContainer.source = "form.pdf";
 
	var tw:GTween = new GTween(this.dataBoardWrapper, 0.25, {x:this.pdfContainer.x + this.pdfContainer.width + 20});
}

So, what the code above does ? Well nothing cool in particular, the _onFocusSendCommand method is called whenever a control in our PDF gets focus, and what the Flex application does is to display an alert message to let the user know the type of data that is expecting for that component. On the other hand, the _onBlurSendCommand gets executed whenever a control in our PDF document has lost its focus and perhaps has some info in it. So whenever that method is executed a cool alert is displayed to tell the user which information has been entered so far in the PDF form. Lastly, the _loadPDF method, well as it name implies LOADS THE DAMN PDF!!! :) it just sets the PDFContainer’s source property to whatever is the PDF URL and invokes the javascript function that loads and displays our PDF object within our HTML page.

To make our PDF document recieve messages from our HTML container is just a matter of adding the proper message handlers, as we did in HTML JS, inside our PDF document javascript code, and call PDFObject.postMessage(whatever) within our javascript. If we set up the proper onMessage handler in our PDF it will be invoked whenever a postMessage is executed in our HTML container. And to add our SWF into the equation is just a matter of adding the proper ExternalInterface callbacks to do. Probably a topic worth a following post, I’ll just need to clean up somethings up and I’ll post it some day.

I hope, if you made it this far, that you liked what you read and found it useful as I did :) There are a lot of cool stuff that could be accomplish way beyond of what was explained here. This is not intended to be a thorough guide on how to interact with your PDFs but to serve as a starting point and give you an overall idea of how you can start to do so. There are a lot of other features that can be pulled off with a Acrobat javascript, again, if you are on to this here you can find interesting documentation.

Thanks for stopping by, your comments, and cheers, now is time for me to grab a beer!


Nov 5 2009

Recording Audio and Video with Red5

What’s up people, these last days (and weeks) I’ve been working with Red5, and learning along the way because I had never used it beyond ‘Hello world’ type experiments, and it’s a shame I did not started to play with it earlier, because is so damn cool. Thanks a lot Red5 Team!

Anyway, I’ve seen some folks here and there asking on how to record a live webcam/mic stream using Red5, and some long ago when I searched for that I did not find a straightforward answer (AKA gimme the code :D ), so for those fellas that are badass lazy as I am, and that their hopes are to find all the help (AKA gimme the code) on Google, here’s a little example that should get your ass going (if you were too lazy to try this suggestion which you should do if you want to really learn )

Ok, I’ll assume you have everything in order so we can go ahead and get our cameras and microphones:

1
2
3
var myCam:Camera = Camera.getCamera();
var myMic:Microphone = Microphone.getMicrophone();
myMic.setLoopback(true);

, then create our net stream and attach the live video/audio to it. Here you could add some validations to verify that you are getting your camera and mic feed as expected (e.g. not null), I will omit it however because my fingers are getting tired.

1
2
3
4
5
6
7
// Create our net stream, by the way for the sake of brevity let's assume you have all your connection sh!t together
var stream:NetStream = new NetStream(myNetConnection);
// add a listener so we know when our stuff is ready for recording
stream.addEventListener(NetStatusEvent.NET_STATUS, onRecStreamStatus);
stream.attachCamera(myCam );
stream.attachAudio(myMic);
stream.publish('myStream', 'live');

Now the stream status event handler, as you’ll see, when your stream has started to “stream” ha! :D then you are ready to record,

1
2
3
4
5
6
7
private function onRecStreamStatus(e:NetStatusEvent):void
{
        if ( e.info.code == "NetStream.Publish.Start" )
        {
             myConnection.call('startRecordingStream', null);
        }
}

This line myConnection.call('startRecordingStream', null); invokes the method ‘startRecordingStream’ on our Red5 service which is responsible for handling our streams and recording/saving them accordingly… and it could look something like this…is Java by the way :)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
//Java
 
public void startRecordingStream(IConnection conn)
{
	String clientId = conn.getClient().getId();
	IScope scope = conn.getScope();
 
	String streamName = clientId + "_" + String.valueOf(System.currentTimeMillis());
 
	try
	{
		ClientBroadcastStream stream = (ClientBroadcastStream) this.getBroadcastStream(conn.getScope(), "myStream" );
		stream.saveAs(streamName, false);
		// You could, if you wish of course, notify the user that the recording has actually started
		// You just have to add the recordingStarted public function on your flash application
		ServiceUtils.invokeOnConnection(conn, "recordingStarted", new Object[]{clientId});
	}
	catch(Exception e)
	{
		// You could notify the user that the recording failed for some stupid reason
		// sometimes things does not go well :)
		// You just have to add the recordingStarted public function on your flash application
		ServiceUtils.invokeOnConnection(conn, "recordingFailed", new Object[]{clientId});
	}
}

And if you want to have a button to stop recording on your flash app., just throw a button and when is clicked invokes a method on your Red5 service for stopping the recording process. Something like this:

1
2
3
//AS3
//assuming this is on a click handler or whatever
myConnection.call('stopRecordingStream', null);
1
2
3
4
5
6
7
8
//Java
public void stopRecordingStream(IConnection conn)
{
	String clientId = conn.getClient().getId();
	ClientBroadcastStream stream = (ClientBroadcastStream) this.getBroadcastStream(conn.getScope(), "myStream");
	stream.stopRecording();
	ServiceUtils.invokeOnConnection(conn, "recordingStopped", new Object[]{clientId});
}

As Dominick pointed out on his blog post, you should get your recorded stream under red5/webapps/YouBadAssApplication/streams.

There are lot of things that have to be done in order to have video/voice application running smoothly, however I hope this helps you in realizing how simple Red5 makes things for us. Let me know if I missed something, if there are other workarounds you have found, or whatever you might wanna say :P

Cheers!


Dec 15 2008

Playing a little bit with APE…part I

Lately I’ve been digging a little bit into the world of data visualization and have found some amazing examples and applications on how you can represent different kinds of information using some very fancy and advanced techniques in order to display your data in a more interactive and less boring way.

I have to say that I rather look at a cute graphic displaying a bunch of bubbles with some little tag in it,that you can drag around, they can bump into each other, play a little bit with, waste some time while in it , than just going through some boring lines of text and numbers. One of the samples I ran into was a really cool constellation visualization application that really caught my attention, as well as a bunch of others mentioned here. I have to say ( and unfortunately accept) that they were way over my head hehe; what I  actually needed to do was something rather simpler and something alike the constellation approach was more appealing to me ( and my technical skills heh ), so I thought I should try some physic engine outhere  to handle the nodes and springs the way I wanted to do it.

After going through the existing ( or at least the ones I found ) physic engines for AS3, I found APE to be the most easy to start with and implement (and intuitive), and yet still a very cool and powerful engine; actually I just took a glance at an example and see what was needed to start playing with it; with APE it is just a piece of cake to make a billiards game for crying out loud! man, I remember all the hurdles I ran into when I did one, with a lot of glitches I have to say, a few years ago (with Flash 5), and I don’t recall an existing physics engine back then.

Anyway, FWIW (less), here I just want to present a little example on how to simple use some particles with spring constraints connecting each other, for all the ones that wants to see how easy is to use this fantastic engine, or wants to know how to start. I use a similar approach for my visualization application. That said, this is very BASIC, but other more complex examples will come, as well as an example of implementing APE with games and that sort of sh!t. I’m also preparing one example using WOW (which is based in APE) and PV3D. I know a few weeks ago I said I was going to give it a try and build my own collision detection for 3D objects, but the hell with, this french guy is already doing some amazing stuff.

Just press down over one of the balls and move the mouse. SWF Main Class Ball Class (nothing fancy, you can come up with something much better)

Cheers :D