Read Apache log files line by line: fileStream.readLine()

18 January 2011

I wanted to read access log files by an Adobe Air application. As these files can get very big, there is no way to load the entire file into memory. Unfortunately, FileStream does not help much because there is no readLine() method like in Java’s BufferReader.readLine().

Googling reveals a nice implementation of Sandeep Gupta: FileStreamWithLineReader. While it works fine for pure ASCII files, it’s bogus for files with multibytes characters like öäü. Also, I’m no fan of extending classes and prefer encapsulation of the FileStream. FileStreamLineReader is slower compared to FileStreamWithLineReader but it works for all characters.

Here’s an example how to use the class:

var accessLog:File = new File("access.log")
var logStream:FileStreamWithLineReader = new FileStreamWithLineReader()
logStream.open(accessLog, FileMode.READ)
while(logStream.lineAvailable) {
	var log:String = logStream.readUTFLine()
}
package 
{
import flash.filesystem.File;
import flash.filesystem.FileStream;
import flash.utils.ByteArray;


public class FileStreamLineReader {
	
	private var _buffer:ByteArray
	private var _bufferSize:uint
	private var _fileStream:FileStream
	private var _lastLineEndPosition:int = 0
	private var _lineAvailable:Boolean = true
	
	function FileStreamLineReader(bufferSize:uint=512) {
		_bufferSize = bufferSize
	}
	
	public function open(file:File, fileMode:String):void {
		_buffer = new ByteArray()
		_fileStream = new FileStream()
		_fileStream.open(file, fileMode)
	}
		
	public function get lineAvailable():Boolean {
		return _lineAvailable
	}
	
	public function readUTFLine():String {
		return readMultiByteLine("utf-8")
	}
	
	public function readMultiByteLine(charSet:String):String {
		var toReturn:String = readLine(charSet)
		
		// the following check is a fix when on windows the buffer reads between the values of
		// 13 and 10, which are used to indicate the end of line
		if(toReturn != null && toReturn.charCodeAt(toReturn.length - 1) == 13) {
			return toReturn.substr(0, toReturn.length - 1)
		}
		
		return toReturn
	}
	
	private function readLine(charSet:String):String {
		const initialReadPosition:Number = _lastLineEndPosition
		_fileStream.position = initialReadPosition
		var adaptedBufferSize:uint = _bufferSize
		while (true) {
			var bytesToRead:uint = Math.min(adaptedBufferSize, _fileStream.bytesAvailable)
			_fileStream.readBytes(_buffer, 0, bytesToRead)
			var currentReadString:String = _buffer.readMultiByte(bytesToRead, charSet)
			var index:int = currentReadString.indexOf('\n')
			if(index != -1) {
				currentReadString = currentReadString.substr(0, index - 1)
				_buffer.clear()
				_buffer.writeMultiByte(currentReadString, charSet)
				_lastLineEndPosition += _buffer.length + 2
				_buffer.clear()
				return currentReadString
			} else {
				_buffer.clear()
				if(_fileStream.bytesAvailable == 0) {
					_lineAvailable = false
					return currentReadString
				} else {
					_fileStream.position = initialReadPosition
					adaptedBufferSize *= 2
				}
			}
		}
		throw new Error("could not find line")
	}
	
	public function close():void {
		_fileStream.close()
	}

}
}


Mac vs PC

22 June 2010

This is just to good to miss on a bloc called “I don’t like computers.”:


Which Flash Fonts you have.

18 February 2010

I was struggling last week with the font “Tahoma” because italic was not shown properly. It turned out that Tahoma cannot display italic type. So what were my alternatives?

And here we go, the list of all fonts on your computer. If you see an error, you run into a Flash Player 10.0 bug. Just update to Flash Player 10.1.


ActionScript/Flex Dependency Injection performance

1 February 2010

I’m currently sneaking around the dependency injection (DI) frameworks Dawn, Swiz and Parsley. None of which I have ever used in production, so take my judgment with care. At first look, the use of Dawn is simplest and most concise and somewhat similar to Google’s Guice which I like. In Swiz, I don’t like the global state used in event handling. Parsley is well documented but a framework that is configurable by XML makes me very suspicious.

Tests

DI depends on reflection on classes which is mainly done with describeType() in Flash. I had in mind that it was a very slow function, but I didn’t know for sure. So I wrote some code to estimate the runtime performance of DI frameworks:

Find metatdata tag [Inject] in different classes:
5050 μs for UIComponent
5150 μs for PersonDetails
700 μs for PersonDetails with DescribeTypeCache
30 μs for SimplestClass
360 μs for describeType XML of UIComponent

2 us to dispatch and handle an event

0.2 μs to call getQualifiedClassName(UIComponent)
1.6 μs to call getDefinitionByName(UIComponent)

0.3 μs to call uiComponent['drawFocus']
0.2 μs to call uiComponent.hasOwnProperty('maxHeight')
0.7 μs to call uiComponent['maxHeight']

9 μs to call Log.getLogger('mx.core.UIComponent')
12 μs to call CreateLogger.withClassNameOf(VBox)

μs = microseconds

You can evaluate the Flash Player performance yourself with the app. The code is available with a right click on the app.

Results

describeType() is really slow. It takes more than 5ms on UIComponent. What surprised me positively is the speed of getQualifiedClassName(). I flipped all logger creation to using getQualifiedClassName() with the helper class CreateLogger.as since.

Conclusion

Dependency injection frameworks make only sense in big projects. Let’s assume a project with 200 different classes that have injected objects, the start up time will increase by 1 second. Knowing that Flex applications don’t fire up quickly, 1 second is surely too long. For BlocStac, I will use DI only in side projects and hope that Flash Player offers faster reflection functions soon.


Amazon S3 + Flash crossdomain.xml + IE7 = Certifacte error

18 December 2008

This is just an other story why I don’t like computers… When the Flash Player loads files from Amazon S3, it most likely crosses the domain border. This means that you have to put up yourself with Flash Player security. If you planned a new security concept for Flash Player and you want to make it as complex and convoluted as you could ever imagine and then, you power this by the factor of 10, you most likely come up with the current Flash Player security concept. Ok, if it hadn’t changed much over the years. But did you know that for example the allowed HTTP header changed in every release! I bet that there are no more than 10 people on the world that understand everything including the implications of the changes over the years.

Whatever, loading the crossdomain.xml from Amazon S3 does not fail because of Flash Player. That’s why loading the file works fine in Firefox but fails in Internet Explorer and Safari. The URL was something like https://images.example.org.s3.amazonaws.com/crossdomain.xml“. It turns out that Internet Explorer does not accept address with “.” for wildcard SSL certifactes like “*.s3.amazonaws.com”. So this story ends with the fact that you should only use bucket names without “.” like https://myimages.s3.amazonaws.com/crossdomain.xml“.


Adobe Cocomo RTMFP enables “Skype” without installation

3 December 2008

Yesterday, the blog of the Adobe Cocomo team announced a limited service for RTMFP. The entry is very informative and straight; so go read it! RTMFP stands for Real-Time Media Flow Protocol and is well explained in these FAQ. From an end user perspective, RTMFP seems to be as relevant as for example the move in imaging from CCD-sensors to CMOS-sensors: Both technologies can deliver great result, but with CMOS it gets much easier over the years and it is also much cheaper.

Hopes are, that RTMPF can lower costs of high bandwidth audio and video conversations mainly between two users while featuring a higher quality than a server based communication. The only party-stopper might be restrictive firewall rules. In fact, if I were an IT administrator I would block all those outgoing connections.

To start a connection between two users, you need something like a dating service. Adobe Stratus does this job exclusively. Once you know each other, the service is only loosely involved with your interaction.

What’s missing at first glance is the ability to store those streams. I think it is the same problem as with direct communication to Cocomo. You can persist objects but no streams. While it’s fine to start with a proprietary protocol, I hope Adobe releases the specs as soon as possible. I hope to see RTMFP soon in Wowza and Red5.


[Bindable] causes Error 1144 for Vector classes in interfaces

28 November 2008

Write up an interface like…

package 
{
	import __AS3__.vec.Vector;
	
	public interface Example
	{
		function get links(): Vector.<int>
		function set links( value:Vector.<int> ): void
	}
}

… and a class that implements the interface …

package
{
	import __AS3__.vec.Vector;
	
	[Bindable]
	public class ExampleImpl implements Example
	{
		public function get links():Vector.<int> {	return null; }
		public function set links(value:Vector.<int>):void {}
	}
}

… and receive an error message for free: “1144: Interface method set links in namespace Example is implemented with an incompatible signature in class ExampleImpl.

A workaround is placing [Bindable] on the property level. And that’s it for this week.


Adobe Cocomo is just right.

17 November 2008

Adobe is heading in the right direction and lowers the barrier for real time communication significantly. Integration of text chat or video into applications becomes for the first time cheap in engineering terms. I could not find any information about the business model around it. I hope Adobe finds an acceptable way for monetization of commercial use.

The package 0.9 you currently get, is fully up to my expectation of a beta release. The numerous examples run within seconds (for those that do not read docs as I do, the room URL is something like http://connectnow.acrobat.com/myAccountName/myFirstRoom). Nowadays, security is a must and the implementation looks just fine. You can delegate authentication to your own application.

Architecture

In the developer guide, you find this overview. It gives you a dictionary of the new terms introduced.

Cocomo architecture

What’s next?

Things I’d like to know and will investigate:

  • I want to authenticate users with OpenID. How can I integrate this with Cocomo?
  • Study the rather long license terms and see how liable it is for free stuff and commercial use.
  • What’s Adobe’s business model for commercial products?

Build your own Feed of Adobe Wiki

8 May 2008

The wiki of Adobe Open Source offers a “RSS feed builder” in which you can specify what is pulled into your feed. For instance, you can focus on the “Flex SDK” space and get all the news about Flex 4. This seems to include all change notifications of the build server (which is nicely called “Build Meister”).

I could not find the link to builder in the wiki, so here it is.


What you should know about DisplayObject’s mask and scrollRect

2 May 2008

While adding cropping to an image editing component, I had to learn a few things about displayObject.mask and displayObject.scrollRect:

  1. mask and scrollRect are mutually exclusive: You cannot use both at the same time.
  2. Grant Skinner has got a nice introduction into scrollRect.
  3. Mask can have any form. Andrew Trice features a nice bundle of samples.
  4. ScrollRect is of type Rectangle and hence, the form is always a rectangle.
  5. Mask does not change the size of the displayObject.
  6. ScrollRect does change the size. Ticore Shih has got the examples.
  7. There is a catch with 6. While scrollRect does change the size, you cannot know when this will happen, really! It might be suitable for a random number generator. UIComponent.calllater() does not bring you anywhere.
    It has been reported as a bug in FP-177 just two weeks ago. Though I am not sure what FP 9.2 Code Complete means and the comment is somewhat unclear to me.
  8. The workaround is either using the dimension of scrollRect or redrawing the DisplayObject as described by Ticore. (At least the code shows it; I don’t exactly know what he says in Chinese…)

Follow

Get every new post delivered to your Inbox.