Reverse engineering AMF3 Vector for Java

Did you ever worked in AS3 with Byte Array and AMF data serialization for transferring object over network ? It really simplify everything. Of course there is a size overhead. But the bottom line is that we can save hundreds of hours by using it.
Since Flash 10, there is one big flaw with AMF: it doesn’t support Vector. Well it’s not true, The FlasPlayer support it, but not what link to it. So you still can serialize Vector object in a byte array and save it in a file for example, but you can’t send it to a web page written in php or a server in Java because there is no implementation of those new « flash 10 features (Vector, Dictionary,…)» in the flex framework. The thing you can do is use an Array instead of a vector, and enjoy the conversion in the Java Hash Map server sie. This is done because of the « sparse possibility of the array » (read my Tamarin Array Analysis).

One more thing: not only it’s not implemented, but it’s not even documented!

tonight is gonna be a good good night

I knew that tonight my wife was going to some bar with a friend and that my two kids would be sleeping after 20h00. So I made a bet with my friends at work that I would reverse engineer the Vector serialization in AMF to implement it in the flex messaging framework in Java. And all that before my wife come back at 3h00.

Well guess what! I made it in only three hours!

How to

So what were the difficulties? In fact it was pretty straight forward.
Vector AMF serialization is using the same protocol used by the other data type.

First, you have the « DataType » byte
Those are the current « supported and documented » DataTypes

undefined-marker = 0x00
null-marker = 0x01
false-marker = 0x02
true-marker = 0x03
integer-marker = 0x04
double-marker = 0x05
string-marker = 0x06
xml-doc-marker = 0x07
date-marker = 0x08
array-marker = 0x09
object-marker = 0x0A
xml-marker = 0x0B

To that, you had a few new types because the Vector class is not considered as one type but as many (4).

Vector<int>-marker = 0x0D
Vector<uint>-marker = 0x0E
Vector<Number>-marker = 0x0F
Vector<Object>-marker = 0x10

The next byte is the Vector length written in the Variable Length Unsigned 29-bit Integer Encoding.
The nice thing is, you don’t have to re-implement the variable length encoding because it’s already present since other class are using it too. (readUInt29())
This number may seem to be always twice the actual vector length but it’s only because there is a flag in the last bit of the number.
So your length is in fact (length << 1)

For the int, uint and Number DataType, its folowed by one byte of (junk ?) and the whole series of numbers written on 4 byte for int and uint 00000000-FFFFFFFF and on 8 bytes for Number using the 8 byte IEEE-754 double precision floating point value.
Creating those 3 types was really easy, you just have to loop for (size) , and push in an ArrayList of the good type (Integer, Long and Double).

The Object Vector was a bit more tricky because they added optimization to prevent classes to re-write the whole class package and variable definition in each object of the vector, so the objects definition differ during the loop..

What they did is pushing the class type « foo » (for Vector.) in the traits of the vector. So directly after the marker and length, you’ve got the classname used by « registerClassAlias(“com.test”, com.test); »
And then you have a series of « length » object all using the same traits. But at this point, there is no class variable declared. Those properties are defined in the first non-null object of the vector and added to the traits.
Since every object is read as a “object” data type, the only thing to be done was to call the “already existing” readObject() function with some management to handle the optimization I was talking about.

Testing

I did some test to make sure everything was alright:

Vector.<int>
Hex: 0D090000000002000007D07FFFFFFF80000000
Original : 2,2000,2147483647,-2147483648
ReadObject : 2,2000,2147483647,-2147483648
Java Read : [2, 2000, 2147483647, -2147483648]

Vector.<uint>
Hex: 0E090000000002000007D0FFFFFFFF00000000
2,2000,4294967295,0
ReadObject : 2,2000,4294967295,0
Java Read : [2, 2000, 4294967295, 0]

Vector.<Number>
Hex: 0F0F003FF199999999999ABFF199999999999A7FEFFFFFFFFFFFFF0000000000000001FFF8000000000000FFF00000000000007FF0000000000000
Original : 1.1,-1.1,1.79769313486231e+308,4.9406564584124654e-324,NaN,-Infinity,Infinity
ReadObject : 1.1,-1.1,1.79769313486231e+308,4.9406564584124654e-324,NaN,-Infinity,Infinity
Java Read : [1.1, -1.1, 1.7976931348623157E308, 4.9E-324, NaN, -Infinity, Infinity]

Vector.<String>
Hex: 10090001060D666F6F5265660106000609666F6F33
Original : fooRef,null,fooRef,foo3
ReadObject : fooRef,null,fooRef,foo3
Java Read : [foo, null, fooRef, foo3]

Vector.<Vector.<int>>
Hex: 100500010D07000000004E00000000000000150D030000000022
Original : 78,0,21,34
ReadObject : 78,0,21,34
Java Read : [[78, 0, 21], [34]]

Vector.<com.test>
10070011636F6D2E74657374010A130007666F6F04020A010403
null,[com.test - foo=2],[com.test - foo=3]
ReadObject : null,[com.test - foo=2],[com.test - foo=3]
Java Read : [com.test@1027b4d, com.test@1ed2ae8, com.test@19c26f5]

In the last test I did class serialization with class alias registration. Here is the code of the “test” class.
The same class is also written in java and the vector is instantiated with the corresponding class!

package com
{
	import flash.net.registerClassAlias;
	registerClassAlias("com.test", com.test);
	[RemoteClass(alias="com.test")]
	public class test extends Object
	{
		public var foo:int = 3;
		public function test() 
		{
			
		}
		
		public function toString() : String
		{
			
			return "[com.test - foo=" + foo + "]";
		}
	}
}

Use it: The Sources

Ok, The work is not final yet, Only the reading is done.. Sigh..
But I plan to do the remaining work this weekend or next week.
The full source is available on my google code:
AMF3 Vector

Since adobe is using “private package“ abstract classes, I had to use their package name (flex.messaging.io.amf) for my classes.

The Main class is a copy of the Amf3Input class use for data deserialization.
I renamed it temporarily “Amf3InputUpdated” and it can be use exactly like the previous one. I also added all the flex-messaging.* libs coming available in the BlaseDS package. You need those to be able to use AMF serialization in Java.

If you come back later on this week, the whole thing should be completed. I will also add the Dictionary class support.

Vector Encode:

Please refer to this:
http://blog.infrared5.com/2011/08/red5-vector-support/

References:

Action Message Format – AMF3