Skip to content

Base64 Optimized as3 lib

January 9, 2010

*Edit january 2012*

A New faster version is available here: http://www.sociodox.com/base64.html

*End edit*

 

A few weeks ago I started a personal project that needed to send binary data to a text (not binary) receiver. This couldn’t be done without encoding the binary data in a format accepted by the receiver. There is a few well know encoder: Base64, yEnc, etc.
Base64 can have a 30% size overhead (compare to yEnc) but it’s fast and easy to use. I chose Base64.

While searching the net for some Base64 lib in as3, I found the class inside the as3Crypto package, and one from the PHPRPC protocol.

So I started with those lib. DAMN it was slow.

So why after a few years of as3 there is still no optimized library for something that common?

I went through both lib and optimized what I could. The final version is ~700% faster with almost no heap expansion compare to both libs.

What’s wrong with current as3 base64 libs

First of all, they are all using string manipulation. WHY?!

private static const BASE64_CHARS:String = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";

By using a String as the character map, you must then use String manipulation such as:

BASE64_CHARS.charAt(outputBuffer[k]);

The as3Crypto is also using bytearray ::bytesAvailable for the main loop which is really slow !

// while there are still bytes to be processed
                        while (data.bytesAvailable > 0) {

For creating the final string, one lib join (array.join()) all the characters pushed in one array.
The other lib (crypto) make String += operation for each character added (it hurts!)

What shouold it be?

Well by definition, base64 is a binary-to-text encoder, but the encoding process have nothing to do with strings!
You start with an 8 bit byte (256) to get to a 6 bit byte (64)
So you take your first 6 bits and put them in one byte and the last 2 bit and put it in another byte.
But if you stop there you have 4 bits lost in the second byte. So we’ll compact this in the least common multiple of 8 and 6: 24!
So you read 3 bytes, modify it and output them in 4 bytes. (size overhead of ~30%)

				//Read 3 Characters (8bit * 3 = 24 bits)
                c = data[i++] << 16 | data[i++] << 8 | data[i++];

Both previous lib were using character by character concatenation.
But as I said previously, no strings in needed here… Its all byte!
So I decided to output the characters in a byte array, and return the final strings with bytearray.readUTFbytes(). (this is very fast compared to both others techniques)
One cool thing about the base64 algorithm is that you know the final length from the start:

out.length = (2 + data.length - ((data.length + 2) % 3)) * 4 / 3;

Optimization: we can set the ByteArray exact length and skip “grow” call and unneeded heap expansion. (Read my previous post concerning heap size)
The algorithm will extract your data into 6 bits chunks, but you still need to map these with the official base64 characters. In a byte array, what is stored is the “value” of that character (‘a’=97) so you don’t need to work with strings at all.
For the character mapping, you need a vector. of all the 64 characters values.

			var chars:String = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
			for (var i:int = 0; i < 64; i++)
			{
				encodeChars.push(chars.charCodeAt(i));
			}

We could write the data in the final buffer with:

				out.writeByte(_encodeChars[c >> 18] );
				out.writeByte(_encodeChars[c >> 12 & 0x3f]);
				out.writeByte(_encodeChars[c >> 6 & 0x3f]);
				out.writeByte(_encodeChars[c & 0x3f]);

But I have seen a lot of slower and older computer have better performances with only one ByteArray write so instead I used some bitwise to create an Int representing the 4 characters, and I write one Int at a time into the stream.

				//Convert to 4 Characters (6+2 bit * 4 = 24 bits)
				c = (_encodeChars[c >>> 18] << 24) | (_encodeChars[c >>> 12 & 0x3f] << 16) | (_encodeChars[c >>> 6 & 0x3f] << 8 ) | _encodeChars[c & 0x3f];
				out.writeInt(c);

On my old laptop, using one writeint instead of 4 writebyte cut the processing time in half!
And the last step before sending the final string is adding (if needed) the base64 padding characters (‘=’)
I won’t talk about the decoding code, but it’s all in the joined source

Conslusion

Don’t work with strings when you don’t REALLY need to. (They are slow)
Don’t use ByteArray methods if you don’t REALLY need to. (They are slow)

Full source code

The Base64 class is now on google code

*Edit january 2012*

A New faster version is available here: http://www.sociodox.com/base64.html

*End edit*

And here it is:

/* Base64 library for ActionScript 3.0.
* Based on: Ma Bingyao code.
* Optimized by: Jean-Philippe Auclair  / jpauclair.wordpress.com
* Copyright (C) 2007 Ma Bingyao <andot@ujn.edu.cn>
* LastModified: Oct 26, 2009
* This library is free.  You can redistribute it and/or modify it.
*/
package
{
    import flash.utils.ByteArray;
    public class Base64
	{
        private static const _encodeChars:Vector.<int> = InitEncoreChar();
        private static const _decodeChars:Vector.<int> = InitDecodeChar();

        public static function encode(data:ByteArray):String
		{
			var out:ByteArray = new ByteArray();
			//Presetting the length keep the memory smaller and optimize speed since there is no "grow" needed
			out.length = (2 + data.length - ((data.length + 2) % 3)) * 4 / 3; //Preset length //1.6 to 1.5 ms
			var i:int = 0;
            var r:int = data.length % 3;
            var len:int = data.length - r;
            var c:int;   //read (3) character AND write (4) characters

            while (i < len)
			{
				//Read 3 Characters (8bit * 3 = 24 bits)
                c = data[i++] << 16 | data[i++] << 8 | data[i++];

				//Cannot optimize this to read int because of the positioning overhead. (as3 bytearray seek is slow)
				//Convert to 4 Characters (6 bit * 4 = 24 bits)
				c = (_encodeChars[c >>> 18] << 24) | (_encodeChars[c >>> 12 & 0x3f] << 16) | (_encodeChars[c >>> 6 & 0x3f] << 8 ) | _encodeChars[c & 0x3f];

				//Optimization: On older and slower computer, do one write Int instead of 4 write byte: 1.5 to 0.71 ms
				out.writeInt(c);
				/*
				out.writeByte(_encodeChars[c >> 18] );
				out.writeByte(_encodeChars[c >> 12 & 0x3f]);
				out.writeByte(_encodeChars[c >> 6 & 0x3f]);
				out.writeByte(_encodeChars[c & 0x3f]);
				*/
            }

            if (r == 1) //Need two "=" padding
			{
				//Read one char, write two chars, write padding
                c = data[i];
				c = (_encodeChars[c >>> 2] << 24) | (_encodeChars[(c & 0x03) << 4] << 16) | 61 << 8 | 61;
				out.writeInt(c);
            }
            else if (r == 2) //Need one "=" padding
			{
                c = data[i++] << 8 | data[i];
				c = (_encodeChars[c >>> 10] << 24) | (_encodeChars[c >>> 4 & 0x3f] << 16) | (_encodeChars[(c & 0x0f) << 2] << 8 ) | 61;
				out.writeInt(c);
            }

            out.position = 0;
			return out.readUTFBytes(out.length);
        }

        public static function decode(str:String):ByteArray
		{
            var c1:int;
            var c2:int;
            var c3:int;
            var c4:int;
            var i:int;
            var len:int;
            var out:ByteArray;
            len = str.length;
            i = 0;
            out = new ByteArray();
            var byteString:ByteArray = new ByteArray();
			byteString.writeUTFBytes(str);
            while (i < len)
			{
				//c1
                do
				{
                    c1 = _decodeChars[byteString[i++]];
                } while (i < len && c1 == -1);
                if (c1 == -1) break;

				//c2
                do
				{
                    c2 = _decodeChars[byteString[i++]];
                } while (i < len && c2 == -1);
                if (c2 == -1) break;

                out.writeByte((c1 << 2) | ((c2 & 0x30) >> 4));

				//c3
                do
                {
                    c3 = byteString[i++];
                    if (c3 == 61) return out;

                    c3 = _decodeChars[c3];
                } while (i < len && c3 == -1);
                if (c3 == -1) break;

                out.writeByte(((c2 & 0x0f) << 4) | ((c3 & 0x3c) >> 2));

				//c4
                do {
                    c4 = byteString[i++];
                    if (c4 == 61) return out;

                    c4 = _decodeChars[c4];
                } while (i < len && c4 == -1);
                if (c4 == -1) break;

                out.writeByte(((c3 & 0x03) << 6) | c4);

            }
            return out;
        }

		public static function InitEncoreChar() : Vector.<int>
		{
			var encodeChars:Vector.<int> = new Vector.<int>();

			// We could push the number directly, but i think it's nice to see the characters (with no overhead on encode/decode)
			var chars:String = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
			for (var i:int = 0; i < 64; i++)
			{
				encodeChars.push(chars.charCodeAt(i));
			}
			/*
				encodeChars.push(
				65, 66, 67, 68, 69, 70, 71, 72,
				73, 74, 75, 76, 77, 78, 79, 80,
				81, 82, 83, 84, 85, 86, 87, 88,
				89, 90, 97, 98, 99, 100, 101, 102,
				103, 104, 105, 106, 107, 108, 109, 110,
				111, 112, 113, 114, 115, 116, 117, 118,
				119, 120, 121, 122, 48, 49, 50, 51,
				52, 53, 54, 55, 56, 57, 43, 47);
			*/
			return encodeChars;
		}

		public static function InitDecodeChar() : Vector.<int>
		{
			var decodeChars:Vector.<int> = new Vector.<int>();

			decodeChars.push(-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 62, -1, -1, -1, 63,
							 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, -1, -1, -1, -1, -1, -1,
							 -1,  0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14,
							 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, -1, -1, -1, -1, -1,
							 -1, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
							 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, -1, -1, -1, -1, -1
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
							 -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1);
			return decodeChars;
		}
    }
}

References:

http://code.google.com/p/as3crypto/
http://ntt.cc/2009/11/28/base64-encoder-and-decoder-with-actionscript-3-full-source-code.html

add to del.icio.usAdd to Blinkslistadd to furlDigg itadd to ma.gnoliaStumble It!add to simpyseed the vineTailRankpost to facebook

From → actionscript

52 Comments
  1. nicely done.

  2. Awesome! I wonder how much faster yet this could be using tools from the Apparat project to optimize the ByteArray manipulation.

  3. ohh my!!! little i know about all this byte stuff but all in all i’m really impressed by your work.

  4. Thanks for this implementation; for those of you who want to manipulate Strings instead of ByteArray’s you can add a couple of cheeky helpers methods:

    public static function encodeString(value : String) : String
    {
    	const source : ByteArray = new ByteArray();
    	source.writeUTFBytes(value);
    	return encode(source);
    }
    
    public static function decodeToString(str : String) : String
    {
    	return decode(str).toString();
    }
    
  5. andyone permalink

    Good version, but not best. How about my more faster and simple implementation – http://pastebin.org/84812.

    My version work without Vectors, and you may use it with 8 and 9 player versions.

    • I got to benchmark this because just by looking at it, doesn’t seem possible to be faster.

      • andyone permalink

        I test. You version faster. You right, sorry.

      • andyone permalink

        More faster :(

  6. I bet if you use readString instead of readUTFBytes it might be faster, given that Base64 doesn’t include any non-ASCII characters, and ByteArray wouldn’t have to try to decode.

    • That probably would work if I wasn’t a moron. I thought there was a way to put the bytes in a bytestring, but it seems that all Flash supports is UTF, which means slightly slower due to decoding.

      Also, you sure you got your endianness correct? I don’t think it has a constant default between platforms (I believe it uses the native endianness) so you probably want to explicitly set it to be LITTLE_ENDIAN.

      Third, AS3 has a hidden syntax for declaring Vectors:

      var n:Vector. = new [1, 2, 3];

      So you could probably eliminate the cinit overhead for that function call for that. If they removed it sometime before release, you could also use a static class block.

  7. wvxvw permalink

    Hi.
    Here’s some kind of a result of a contest me and my friend had on making Base64 decoder/encoder:
    http://code.google.com/p/e4xu/source/browse/trunk/haxe/src/org/wvxvws/encoding/Base64.hx
    (My friend won in the end, though the difference is minor). If you compare this to as3corelib, then, according to my tests it’ll be about 20-25 times faster when encoding and 10-12 times faster when decoding. Of course the results may vary in different players / OS, so I don’t claim that to be the very precise estimation, though, it must be faster ;)
    This may not be a polished version, however, if you like it, of course, you can use / modify it.

    Best.
    -wvxvw

    • This is a “best-case” for fast mem opcodes for sure.
      I wanted to make a pure as3 compatible file but it’s great that you made one with haXe.

      Thanks a lot for sharing!

    • One thing I wanted to try with the fastmem opcodes was to read 12 byte at a time (3 int) and convert those to 16 bytes (encoded). it’s not possible with standard as3 because of the cost of the “bytearray.position”. But it would be possible to try it with the fast opcodes.

  8. Mr. Entropy permalink

    Would you mind posting a few use case examples?

  9. Big Brother permalink

    Yes, a few examples would be helpful. I’m getting 1046 errors when I try to create an instance of the Base64 class…

    Scene 1, Layer ‘Layer 2′, Frame 1, Line 604 1046: Type was not found or was not a compile-time constant: Base64.

    • I guess if you’re having this many errors it’s because you classpath does not include the Class.
      The class use all static function so it’s very easy to use.

      Include the source path in your project,
      Import the class Base64
      Base64.encode(*valid byte array to encode*)
      Base64.decode(*string previously encoded*)

  10. Great stuff bro!

  11. Does this worked with compressed byte arrays using bytearray.compress()? I’ll be using this if it does or not, but I’d like to save myself from any headaches by asking here first.

  12. Neat – I’m in need of these exact feature, so thanks for making this publicly available!

    You might want to check out flex.graphics.ImageSnapshot. It does half the job and it’s what I’ve used up until now.

    J

  13. Something like this for initialization seems shorter…

    private static function EncDecChars():String
    {
    return “ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/”;
    }

    private static function InitEncodeChar() : Vector.
    {
    var encodeChars:Vector. = new Vector.(64, true);
    var chars:String = fEncDecChars();
    for (var i:int = 0; i < 64; i++)
    encodeChars[i] = chars.charCodeAt(i);
    return encodeChars;
    }

    private static function InitDecodeChar() : Vector.
    {
    var i:int;
    var decodeChars:Vector. = new Vector.(256, true);

    for (i = 0; i < 256; ++i)
    decodeChars[i] = -1;

    var chars:String = fEncDecChars();
    for (i = 0; i < 64; ++i)
    decodeChars[chars.charCodeAt(i)] = i;

    return decodeChars;
    }

  14. Thanks so much for sharing this. And good add by Johnny Reeves, it’s just what I needed. :)

  15. Sergio permalink

    Thanks a lot for this jpauclair.
    Nice done, you saved my day… Cheers from Mexico.

  16. Your blog is seriously valuable. Thank you very much for maintaining a plenty of First-class post.

  17. flashyhawk permalink

    Reblogged this on Flashy Hawk.

  18. I need to send Base64 that I encoded in flash to server .
    Is there any class can decode it to byte array in dot Net C# because current dot net library cant decode this Base64

  19. it will be nice to have one or more examples. tanks anyway

  20. Does your website have a contact page? I’m having problems locating it but, I’d like to send you an email. I’ve got some creative ideas for your blog you might be interested in hearing. Either way, great website and I look forward to seeing it improve over time.

  21. Anton permalink

    thank you very much!

  22. Howdy! I simply want to give an enormous thumbs up for the great information you have here on this post.
    I will likely be coming back to your blog for more soon.

  23. Hi there to every one, it’s in fact a nice for me to go to see this web page, it includes useful Information.

  24. install whatsapp on pc permalink

    Aw, this was a really good post. Finding the time and actual effort to
    produce a superb article… but what can I say… I procrastinate a whole lot and don’t seem to get anything done.

  25. Great items from you, man. I’ve have in mind your stuff previous to and you’re simply extremely excellent.
    I actually like what you have received right here, really like what you’re stating and the way in which during which you say it.
    You are making it enjoyable and you still care for to
    keep it wise. I can not wait to learn far more from you. That is really a great site.

Trackbacks & Pingbacks

  1. uberVU - social comments
  2. AS3 hidden treasure in the mm.cfg file. Revealing and documenting many Flash secrets! « jpauclair
  3. links for 2010-02-18 – Confessions of an Flash Addict
  4. Flash Player 10.1 – Camera support with PhoneGap | www.flashmobileblog.com
  5. Flash Assembler (not ByteCode) : And you thought it couldn’t get faster « jpauclair
  6. jpauclair.net: 6 Months – Articles recap. « jpauclair
  7. Weekly Digest for June 20th — Hello. My name is Václav Vančura.
  8. .: FlashMX.US :. Blog » Convert ByteArray to a POST variable
  9. Decode Basis64 ByteArray - Flashforum
  10. Embedding Pixel Bender files in AS3 without craziness (or Flex) | Galiard's Last Stand -- A Daunting Puzzle Game
  11. Bringing Flash Video to HTML5 Canvas | Maschinenbordbuch
  12. Preset brush sharing with base64 encoding » Danik Games DevBlog
  13. Signing an Ooyala API query with AS3 | FlexDaddy
  14. Base64 image encoding in Adobe Air | ViToBlog
  15. Updated the optimized Bas64 library « jpauclair
  16. Superfast Base64 Actionscript Library | K2xL
  17. Reverse engineering AMF3 Vector for Java | Parabolic Evidence

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 33 other followers

%d bloggers like this: