Eventual Consistency

Developing in Android without Eclipse

2009-09-27T04:44:00.000-07:00

Recently I'm back in java land, more specifically android. The past few months I've exclusively been doing scripting languages using much more primitive editors (textmate, emacs). Going back to java, I'm finding Eclipse is just getting in my way. So I decided to try and develop an android app using textmate and found it incredibly easy. It appears you have everything you need from the command line tools and all the eclipse plugin does is interface with those.

Step 1, create your project

THis couldn't be easier:

android create project --target 2 --name install_test --path . --activity InstallTest --package com.jonandkerry.install

Obviously make sure your android sdk is in your path. NOTE: if you are using SNow leopard you will need to patch your android install. See here

Step 2, build your project

The android create project tool actually builds you a very simple java skeleton project with an ant build.xml. You get several targets:

aidl
android_rules.aidl
android_rules.compile
android_rules.debug
android_rules.debug-sign
android_rules.dex
android_rules.dirs
android_rules.help
android_rules.install
android_rules.no-sign
android_rules.package
android_rules.package-resources
android_rules.release
android_rules.release-package
android_rules.release.check
android_rules.release.nosign
android_rules.resource-src
android_rules.uninstall
android_rules.uninstall.check
android_rules.uninstall.error
compile
debug
debug-sign
dex
dirs
help
install
no-sign
package
package-resources
release
release-package
release.check
release.nosign
resource-src
uninstall
uninstall.check
uninstall.error
Default target: help

obviously you see and "ant compile", "ant install", etc. Its important to note that ant compile will generate the R.java resource just like eclipse does.

Step 3, run

Couldn't be easier.

$ ant install
Buildfile: build.xml
[setup] Project Target: Android 1.6
[setup] API level: 4

dirs:
[echo] Creating output directories if needed...

resource-src:
[echo] Generating R.java / Manifest.java from the resources...
[exec] (skipping hidden file '/Users/jonathan/Development/personal/install_test/res/.DS_Store')
[exec] (skipping hidden file '/Users/jonathan/Development/personal/install_test/res/layout/.DS_Store')

aidl:
[echo] Compiling aidl files into Java classes...

compile:
[javac] Compiling 1 source file to /Users/jonathan/Development/personal/install_test/bin/classes

dex:
[echo] Converting compiled files and external libraries into bin/classes.dex...

package-resources:
[echo] Packaging resources
[aaptexec] Creating full resource package...
[null] (skipping hidden file '/Users/jonathan/Development/personal/install_test/res/.DS_Store')
[null] (skipping hidden file '/Users/jonathan/Development/personal/install_test/res/layout/.DS_Store')

debug-sign:

package:
[apkbuilder] Creating install_test-debug-unaligned.apk and signing it with a debug key...
[apkbuilder] Using keystore: /Users/jonathan/.android/debug.keystore
[apkbuilder] /Users/jonathan/Development/personal/install_test/bin/classes.dex => classes.dex

debug:
[echo] Running zip align on final apk...
[echo] Debug Package: bin/install_test-debug.apk

install:
[echo] Installing bin/install_test-debug.apk onto default emulator...
[exec] 317 KB/s (10728 bytes in 0.032s)
[exec] pkg: /data/local/tmp/install_test-debug.apk
[exec] Success

BUILD SUCCESSFUL
Total time: 5 seconds

And there you go. To be honest, I'd much rather run this stuff from the command line than watch the spinning beach ball in eclipse.

For more things you can do, see the official documentation:

And also, there is a textmate plugin that simply runs a few of these tasks:

Installing packages in Android

2009-09-25T13:19:00.000-07:00

I've been playing around with Android quite a bit lately. One really cool thing is you can write a package (package == application) that can install other applications. Obviously this isn't done silently. A UI will be presented to the user asking them if they want to install this package and info about it.

Its basically done by sending an intent.

Intent intent = new Intent(Intent.VIEW);
intent.setDataAndType(Uri.parse("The url to your package"), "application/
vnd.android.package-archive");
startActivity(intent);

And there you go. Obviously of you want to to be notified when its complete you can also do a startActivityWithResponse call. I see this as being very powerful.

Interesting opportunities

2009-07-27T02:53:00.000-07:00

On Monday Techcrunch released this story:

Justin.tv Opens Its API For Free, Hopes Live Video Will Explode - http://shar.es/xYpF

I don't have much experience in JustinTV except for the few events I've been to that have used it. From my understanding its like a YouTube but live video rather than uploaded and downloaded. I can't help but see the opportunities in this.

Venture Capital Investment Stabilizes, sort of

2009-07-21T01:32:00.000-07:00

Venture Capital Dollars Stabilize in Second Quarter at Mid-1990s Levels

The above article shows the first increase in VC deals in a while. While the internet and clean tech are still floundering it appears VCs are all about biotech and medical devices these days. I knew I should have paid more attention in biology.

Joining technorati

2009-07-14T11:24:00.001-07:00

Yes I'm joining technorati, ignore this

xfkq4bzcgs

Apache Camel TLP at Apache

2009-01-20T07:09:00.000-08:00

Apache Camel was just upgraded to a top-level project. If you haven't played around with Camel you really should check it out. It abstracts any jms messaging bus and gives you a pretty interface to use cookie-cutter enterprise integration patterns. When looking at some of the patterns alarms go off on how many times I've had to hack them out on my own. Great tool to have in your pocket.

http://camel.apache.org/

Cherrypy + Routes with mod_wsgi

2008-12-02T19:10:00.000-08:00

If you haven't played around with deploying your python apps using mod_wsgi, I highly recommend it. It is a very stable way to deploy your application that takes advantage of the good parts of the apache2 runtime.

Vanilla Cherrypy apps run great as wsgi applications. However, I did come across a problem when deploying a Cherrypy app with Routes. Running the application using the cherrypy server worked fine, but deploying it using apache2 + mod_wsgi always gave me the following on every request:

503 Service Unavailable

The CherryPy engine has stopped.

After some looking around the mod_wsgi project (which is well documented I must say) I found the answer here: http://code.google.com/p/modwsgi/wiki/IntegrationWithCherryPy

When running a Cherrypy app with the routes dispatcher you need to make sure you call:

cherrypy.engine.start(blocking=False)

After mounting your app. This sets the cherrypy engine state to running. After doing this everything worked great and I now have beautiful restful routes in my mod_wsgi app.

REST with cherrypy

2008-11-21T18:15:00.000-08:00

I've been doing web services for a while. And while SOAP definitively has its place, if you control both sites of the web service REST can really make life easier. Cherrypy with Python Routes pretty much makes this trivial. Obviously everyone knows how to route gets, but what about the nasty post and put (REST talk for create and update). NO problem.

d = cherrypy.dispatch.RoutesDispatcher()

d.connect('create_something', '/something', controller=root.something_controller, action='create_something',conditions=dict(method=['POST']))

What this is saying is I want a to route all routes that match /something and are of the method "POST" to the controller root.something_controller and the method "create_something". Thats it. PUT is just as easy.

d.connect('create_something', '/something/:id', controller=root.something_controller, action='update_something',conditions=dict(method=['PUT']))

Same thing as above except the method update_something will get the param id.

routes with Cherrypy

2008-10-05T05:23:00.000-07:00

As I posted earlier, my new favorite web framework is cherrypy. I love its simplicity and the fact that I'm not wrestling with magic to do exactly what I want to do. One bug knock against cherry py is its routing mechansim. By default it uses an object reference routing, so take the following:

class HelloWorld()
def index(self):
return "

Welcome

"

def say_hello(self):
return "

Hello World!

"

class Root(object):

hello = HelloWorld()

app = cherrypy.tree.mount(Root(), config=conf)
cherrypy.quickstart(app)

Notice that our Root object creates a controller as a member var called hello. By default the way we would have a browser call the "say hello" method is by:

http://127.0.0.1:8080/hello/say_hello

This is fine and simple, but as we all know in the world of search engine optimization you have to put as much into your urls as you do your content. This is why things like swimlanes and routes in frameworks such as rails and Servlets are so important. Not to fear becase you can do this in cherrypy also.

Cherrypy pretty much allows you to drop in any request dispatching mechanism you want. Thats the whole point of cherrypy is it completly stays out of your way so you can build the exact webapp you want. If you want to use the python routes package, go a ahead. If you want to write you own, more power to you.

Here is a tutorial on using routes with cherrypy.

Quickly, just by adding the following lines to the code where I start my cherrypy app:

 d = cherrypy.dispatch.RoutesDispatcher()
d.connect('blog', 'helloworld/, controller=Root.hello, action="say_hello")

And I now get routes style power!

python and sweet cherrypy

2008-10-02T03:41:00.000-07:00

I've been doing a lot of python sever-side programming lately, and I really like it. One diamond in the rough I've found is Cherrypy (http://www.cherrypy.com). Cherrypy is a very lightweight and pythonic web framework that does the bare-minimum needed to get you up and running with a model/view controller framework. After that its all up to you and it pretty much stays out of your way.

At its basic cherrypy is really just an application. They do include an http server that runs pretty well, and you can also run your app as a mod_wsgi or mod_python app.

Lets write hello world:

import cherrypy

class HelloWorld(object):
  def index(self):
      return "Hello World!"
  index.exposed = True

cherrypy.quickstart(HelloWorld())

running this piece of code you see:

[02/Oct/2008:06:55:40] ENGINE Listening for SIGHUP.
[02/Oct/2008:06:55:40] ENGINE Listening for SIGTERM.
[02/Oct/2008:06:55:40] ENGINE Listening for SIGUSR1.
[02/Oct/2008:06:55:40] ENGINE Bus STARTING
CherryPy Checker:
The Application mounted at '' has an empty config.

[02/Oct/2008:06:55:40] ENGINE Started monitor thread '_TimeoutMonitor'.
[02/Oct/2008:06:55:40] ENGINE Started monitor thread 'Autoreloader'.
[02/Oct/2008:06:55:41] ENGINE Serving on 127.0.0.1:8080
[02/Oct/2008:06:55:41] ENGINE Bus STARTED

and going to http://127.0.0.1:8080/ you see "hello world". 

Thats pretty much it.

I know I know, a lot of you are saying "Wait, why would I use this over something like JEE and rails that does all this majic for me?" There is no shortage of "out of the box" web frameworks that promise a "build your app in hours" magic

I've been using web frameworks going on 9 years, everything from JEE (Jboss), Ruby on Rails, and even Django. And they are all great. The probem is they are opinionated. To try and remove responsibilities from the developer they make decisions for you, for example:

How they create and commit your db session between requests
How many db sessions you can have.
View rendering technologies
which ORM to use

The list goes on and on. And if you are using one of these frameworks you are enjoing all these "freebies" if you agree to follow the opinion and rules set by the framework. However, the moment you have to go against one of these "pre-decisions" you can find yourself wrestling with the framework. Just try to extend RoR to have multiple db sessions. Or JBoss to use a "share nothing" architecture. I usually find these issues come out when addressing scaling issues specific to your user behavior.

With cherrypy's "minimal" architecture you are free to pretty much do what you want, but you have to do it. You decide how db sessions are opened and closed before and after requests, or how your controllers are stuctured. Its all up to you. And the interesting thing is when you start doing many of the "freebies" your self, you find you you weren't getting as much for free as you thought.

closures and you

2008-04-25T20:46:00.000-07:00

So I'm diving into dynamic programming with Groovy and I love it. I consider myself an expert Java & C++ programmer, but I'm definitely having to do some re-learning with dynamic languages. I think its like when you encounter someone who, for some reason or another, really doesn't grasp object oriented methodologies. Its a way of thinking and designing your software that you really just have to understand. I think programming in dynamic languages, while very different, follow the same learning process. And I'm definitely in it.

So lets start with closures, probably the most verbalized features of dynamic languages.

Obviously the most famous is iterating a list. Lets say I want to print out every element in a list... obviously I could do the following in straight java.

Vector list = new Vector();
// add stuff to the list
for ( Object x : list)
System.out.println(x);

In groovy I can do:

def list = ["one","two","three"]
list.each { println ${it}}

The idea is I actually pass the list a segment of "code" to execute on itself. But wait, it gets better.

Lets say I want to create a function that will do something n times... easy in Groovy:

def doSometing(number, Closure c) {
0.upto(number) {c}

}

// now you can call the method like this:
doSomething (4) { println "Say Hello" }

Pretty smooth huh? Obviously the possibilities are endless and, possibly dangerous. When passing a closure to an object, you basically give it permission do do whatever it wants

class Hello {
String name
String important = "DON'T CHANGE"

def Hello (name, Closure c) {
this.name = name

c()
}
}

I can then create a Hello object like this

def h = new Hello("Jonathan" ) { this.important = "BAD"}

And the Hello object's important field will be changed, even if I make it private. Makes it a little difficult to create well-formed and protected libraries.

grails and the nulls that confuse me

2008-04-17T20:13:00.000-07:00

Recently I've started doing a lot of my work in groovy. Being an old fart I love my java, but many of the young guys are pushing me to pick up the dynamic languages of the times. I'm not a fan of Ruby (mostly due to the community, you know who) and while I like Python it just doesn't fit for scalable enterprise systems.

Then I noticed groovy, and I love it. I have all the power of java with the syntactical candy of dynamic languages. And Grails is really awesome (especially the 1.x series). Its like Ruby on Rails, but made by people who really want to use it in production. I can write groovy code, and use hibernate, and run on top of a clustered Jetty with terracotta, how cool is that?

I do have to say, its the little things that get you. A good example is GORM and exception handling. In EJB3 or even straight hibernate, when I save an object that doesn't validate I'll get a java.lang.RuntimeException. Example

Person p = new Person();
p.setName("something Bad);
entityManager.persst(p);

If for some reason p isn't able to be saved by the entity manager a runtime exception will be thrown. Being a seasoned EJB developer I'd use this exception to my advantage and let it trickle out to the container and roll back the managed transaction (I think this is beautiful, but I guess a thing of the past).

Groovy, grails, and even rails does this different. The way you know an entity is saved is by the return value on the entity.save() method. Lets use the example above in grails.

def p = new Person(name:"something bad")
p.save()

if for some reason p is unable to be saved we won't be notified by the above code, you actually have to do it like this:

def p = new Person(name:"something bad")
if ( !p.save()) { p.errors.each(log.error(it))}

What I find fascinating about this example is a statically typed language is actually less code than a dynamic one!

Jetty & spring are awesome

2008-01-31T18:06:00.000-08:00

One reason I love java is it has so many tools out there to solve the problems you need to solve. I needed to write a server-side app that was way under the scope of jBoss, so I downloaded Jetty and used spring.

For those who don't know, Jetty is a super fast and light weight servlet container that uses NIO heavily. For apps that don't need stuff like transactions its perfect. And pair that up with Spring and you have a super fast & lightweight app that still has very good design.

So check out Jetty and Spring.

Call for Action

2007-11-17T08:26:00.000-08:00

I've worked on a few open source projects and I have nothing bad to say about any of them. If there is one shortfall in the open source world its that everyone wants to be the hero. Everyone wants to "start" an open source project. But far less people are willing to join existing projects and take them those last tedious few steps necessary to truly make them useful. These steps usually include bug fixing and documentation. This is why sourceforge has hundreds of thousands of projects that will never be used.

At RubyConf Charles Nutter issued a "Call for Action" asking people to help take things those last steps, both on documentation of the new Ruby runtime AND bug fixing for JRuby. So if you are interesting in doing something new, try and help him out. Its the little things that makes most open source projects great.

Charles Nutter's Call to Action Post

JRuby and Seam

2007-11-16T19:09:00.000-08:00

After spending a month or so playing with Ruby and Rails I'm really not happy with Rails. I build systems that have to scale and Rails has some serious issues when it comes to that. Its great for simple CRUD apps, but if you need to move outside of that or need to scale its got issues.

I do, however, like JRuby. I'm also a big fan of jBoss Seam. It just makes sense to me, use the scalability of EJB3 with state management in web apps. In JBoss Seam 2.0 you can write your Seam components not only as EJB3, but also in Groovy. I love EJB3 so that would be my first choice. But I work with people who really want to use Ruby. To meet in the middle I've been playing around with integrating JRuby into Seam. Luckly, Seam is designed well to allow this with some learning curve.

I'll post more later, but I've got a proof of concept where I can write my view in JSF, and a Seam-managed component in JRuby, which then uses JPA for persistence.

Dreams of Fuel Cells

2007-11-14T18:29:00.000-08:00

I live about an hour from the auto-motive capital of the U.S. (well, the former one at least). The big 3 and the UAW are pretty much house-hold names in Michigan. Most people who know me would testify that I'm not a fan of the U.S. auto industry. From union squabbling to companies who produce non-creative, sub quality products and insist that the reason you should buy them is not because they are better, but because its your patriotic duty.

Even though I rarely achieve this, I always strive to be a creative, forward thinking engineer. I believe that sometimes its not always enough to just solve a problem, but solve it in a fantastic way that yourself and others can learn from. I simply don't see this a lot from the big 3, which is why I drive a Honda.

Tonight I saw a commercial for the new Honda hydrogen fuel-cell car, and I was captured. I felt like I was looking at the iPhone of cars. A company whose industry is as old as some governments refuses to subside into the monotonous productions of U.S. big business and big labor competitors.

I need to do more reading to fully understand what this car is, but it looks like it will hit the road in the summer of 2008. While other companies are just trying to stay a float and squabbling about overtime pay for employee uniform changes, Honda is making history.

http://www.honda.com/fuel-cell/

well-defined api

2007-11-02T20:47:00.000-07:00

So from being a java/c++ developer for years I've been taught the importance of exposing well-defined save apis. You expose parts of your api that you think is safe for black-box users to incorporate and abstract/protect the rest. This is usually done by the visibility, security, and overriding features of the language.

For instance, if I have a method that I think its unsafe for any class, including sub-classes to call, I'll define it as private.

Ruby seems to think this isn't important. I can do this:

class Base
def aMethod
puts "A private method";
end
private :aMethod
end

class Derived
public :aMethod
end

In one line the sub class "Derived" was able to change the visibility of the method "aMethod". A little scary when trying to define things like data structure libraries and application frameworks that depend on the validity of certain methods.

Ruby on.... jetty?

2007-10-29T18:50:00.000-07:00

So I'm playing around with JRuby, and I think its really cool. The java community has really come a long way and created some great tools for doing enterprise deployments, why should the ruby community spend time backtracking and re-inventing the wheel? The JVM has become a proven execution environment and with an open byte-code standard its trivial to compile to it (well not trivial, but do0-able).

So with JRuby you can get the syntactial sugar of ruby with the heavy lifting of such java services and JMS, hibernate, and even servlet containers. Currently Ruby on Rails uses either WeBrick or Mongrel as their web server. Both are written in ruby and, well, slow. If there is one thing the java community has done a good job of the past few years its build fast and reliable servlet containers (just look at tomcat and jetty).

What I'm wondering is how hard would it be to replace mongrel/WeBrick with jetty in a JRuby on Rails app? Just think of the performance gain with Jetty's NIO engine?

sequel to the rescue

2007-10-29T11:05:00.000-07:00

No sooner than I pushed the "post" button on my last post did I find sequel. Its a light-weight ORM in Ruby that looks promising. Does all the association fetching and even has 2nd-level cache support.

http://code.google.com/p/ruby-sequel/

Active Record, the sludge in Ruby

2007-10-29T06:25:00.000-07:00

So I've been learning Ruby lately, and being a bug ORM guy I decided to checkout the source for active record and see how it works. There are some very serious issues here. Active record uses the dynamic mixin ability of Ruby to do its work, but that same logic kills itself in performance. To explain myself further I we should compare the active record model with JPA.

JPA
Java persistence architecture delegates all ORM work to the persistence manager, an external library that manages db entities. If I want an entity I ask the persistence manager for it, if I want to save, I do it through the persistence manager. When my app starts up the persistence manager looks at all my entities and my schema and creates the necessary object structure to do its ORM magic. The mapping/querying logic is done once and used by everyone.

ActiveRecord
Active record delegates all ORM work to the entity itself. Entities must know how to query/save themselves and if they are transient or persistent. They do this by mixing in methods to to the ruby-defined entities at runtime and then, when an instance of an entity is instantiated, created the necessary links to do its ORM stuff.

The disadvantage of active record is the function of ORM processes is spread throughout however many entities you have. It becomes hard to do things like locking, transactional caching and even transactional enlistment. If I have a 100 entities, each entity has to be concerned with cache invalidation and managing their relationships.

Until the ruby community either fixes this in active record, or comes out with a JPA/Hibernate -like ORM they will never be in the spotlight. If you talk to any ruby-evangelist they always bring up that Twitter uses RoR. And that is true, but if you read any of their posts to scale Twitter had to remove ActiveRecord from their deployment because it was just too slow.

So I think Ruby is cool but I really don't like active record. Anyone interested in porting hibernate to Ruby???

Netbeans and Ruby

2007-10-26T21:08:00.000-07:00

I was working on a Ruby project I'll be revealing later, and I was looking for an IDE. I thought I could get by using VI but after a few years of using Java with Eclipse I've been spoiled. After a little looking I found the Netbeans 6 Beta. They have Ruby package that is fantastic! It contains auto-completion, tool tips, and helpers. Make sure you download the version with the Ruby package installed.

http://dlc.sun.com/netbeans/download/6.0/milestones/latest/

If you are writing Ruby code its well worth the look.

C++ Garbage

2007-10-21T09:39:00.000-07:00

So I've been having conversations about smart/auto pointers and c++ with some of my friends. We all love java and other more modern languages that have garbage collection but there are times when you need to move back into C/C++ land, and rightfully so. I'm not one of those "java/ruby/python is the language to replace all other languages." I use the best tool for the job.

So anyways, one of the most painstaking things about C++ is its absence of garbage collection. Sure, you can always just create things on the stack and pass-by-value but that has its own issue (both in performance and logic). Smart and auto pointers help but don't always give the performance that java gc does. One of the nice things about the GC in java is it is not tied to the owning thread. When an object no longer has any reference to it that thread doesn't block while the object is destroyed. The garbage collector runs in the background, even in parallel, collecting objects smartly that can be destroyed and only doing the work it thinks it can do without consuming too many resources. If I suddenly un-reference or move out of scope a million objects the VM won't necessarily destroy all million right then. It will, over time, destroy them when it thinks it won't hurt performance.

So back to C++. Using C++ templates it isn't difficult to do reference counting on your objects. So what we really need to implement is a singleton garbage collector/reference manager that would keep track of all managed objects, and when they are no longer referenced would mark them to be removed, and then remove them when it thinks its best.

Part 1 Singleton Reference Manager
So first we would need to create a singleton C++ class:

class ReferenceManager
{
public:
   static ReferenceManager* Instance();
protected:
   ReferenceManager();
   ReferenceManager(const ReferenceManager&);
   ReferenceManager& operator= (const ReferenceManager&);


};


Now, if you have never done singletons in C++, the way we insure that one and only one gets created/destroyed is using a local static:

ReferenceManager* ReferenceManager::Instance ()
{
 static ReferenceManager inst;
 return &inst;
}

This way a local static instance is created the first time this method is called, and after that the same instance is used. Its also important to note that since inst is local static, it will automatically be destroyed when the application terminates.

References can be obtained like this:

ReferenceManager *p1 = ReferenceManager::Instance();
ReferenceManager *p2 = p1->Instance();
ReferenceManager & ref = * ReferenceManager::Instance();

Part 2. Reference Counting:
In my next post I'll create my own smart pointer to do reference counting and communicate its state with this reference manager. I would do that now but its beautiful outside and the lawn needs mowing.

Avoid locking with JMS

2007-10-18T07:08:00.000-07:00

One of the hardest things to do in a high-load system is reporting. Sometimes the amount of data you have to keep track of is too big.

Example:
I have a page that is going to be displayed, and need to :

Log every view of the page individually.
Keep a running total of everytime a page is viewed.
Have a max number of times a page can be viewed, and when that has been reached stop showing the page.

Now lets assume that we need to keep all these values in a database so we can scale-up our app server. For requirements 2 & 3 we need to keep an aggregate table with a column that represents a "counter". Without this table we would have to do an aggregation query on the log table (1) to see how many times this has been viewed, as our data grows this will get slow.

So we create an aggregate table table that contains the page name and a counter of how many times that page has been displayed. Every time that page is viewed we insert into the log table and update the counter in the aggregate table.

Locking is a b$$ch

There is a fundamental flaw with the above solution, if I get 100 concurrent page views, the database will have all those page views "lock" on that row in the aggregation table, this is so the db can provide ACID. This will significantly decrease our ability to scale. You could lower your transaction isolation in your db, but you aggregate value consistency may suffer.

JMS to the rescue

A good solution is to use JMS. When a page request comes in we still insert a row into the log table, but with the same XA transaction we also publish a message onto a durable JMS destination. No locking will occur. Now have a consumer (ether a message-driven-bean or a plain jms message consumer) will consume those messages with a durable subscription and update the aggregate asynchronously. Who cares if they lock, it will be on a completely different thread thats not affecting the performance of serving the page.

Obviously its possible to go over your max show variable (race condition that a page is served before the message consumer has a chance to process and update the aggregate) but when you get to high-yield systems you have to start playing payoffs.

So the moral of the story: JMS can play a big role in high-yield systems with its durability and async processing. Next time you have a locking issue or find you are doing too many things during a request, try moving those things behind a durable JMS topic, it may save you some time!

Python and SQLAlchemy

2007-10-11T07:04:00.000-07:00

So if you read this blog its no secret I'm a fan of Hibernate. Not because I'm a Java evangelists but really because I have yet to see many of Hibernates features in other ORMs. Hibernate has gone above and beyond to make an ORM that is not only focused on pojo-based database access but also scalability and durability.

You can tune hibernate to do association fetching and second-level caching that meets your load needs and reduce the amount of bottleneck your database is. You can do projection to limit query context. You can also enlist hibernate in java's JEE global transaction to make it durable and ACID compliant. Many of the other languages and ORMs seem to focus more on developer convenience than these things.

In the python world my boss pointed me to SQLAlchemy. Its a python ORM that is based highly on the design of Hibernate. I like what I see, the association mapping and querying is very similar and powerful.

One thing I see that is missing is the powerful second-level caching concept. Post people's response to this is "Oh, use memcache." Well sure but you loose the acidic properties of your database and ORM. Developers are tasked with first checking in memcache for their entitys, then going to the db, and the same with saving. Both of which don't partipicate in the database transaction they are working in. However, after looking at SQLAlchemy a little more I wonder how hard it would be to build transparent transactional enlistment of it with memcache.

Just think, you decorate your SQLAlchemy object with some sort of marker saying that you want it to participate in second-level caching. Then, along with setting up your persistence context (DB access, etc) you also setup a memcache resource. Now when you access and save your SQLAchemy entities it also pushes and pulls them out of memcache behind the scenes and honors the database ACID transactions. This could do wonders for the scalability of python.

This smells like an open source project to me! Comment with ideas and directions.

JEE isn't just about ORM

2007-10-07T19:23:00.000-07:00

I had a conversation with a friend about JEE and how they thought it can be overkill. I should preface by saying I do believe there are situations where JEE is overkill, which I won't go into. His argument is if you don't need to access a database with an ORM then you don't need JEE.

I strongly disagree with this, mostly because I do this all the time. I love Hibernate and EJB3 entity beans, but it doesn't work for all schemas. If you have a highly normalized relational schema where you can take advantage of 2nd-level caching then entity beans is a blessing. But if you are recording stats and doing more OLAP stuff then you may want to stick with straight JDBC.

However, JEE works great with JDBC also. In fact most deployments I've done, including one I'm currently responsible for, use both. Take the following example

public @Stateless class PersonStatsBean implements PersonStats {

@Resource(mappedName="java:/LoginStatsDS")
DataSource loginStats;

@PersistantContent
EntityManager em;

public void recordLogin(int person_id) throws JDBCException
{
Person person = (Person)em.find(Person.class,person_id);
Connection con = null;
try {
con = loginStats.getConnection();
PreparedStatement stmt = con.prepareStatement("insert into login_stats(person_id)
values (?)");
stmt.setParameter(person.getId());
stmt.executeUpdate();
con.close();
}

}

In this example I am saving a row that keeps track of every time a user logs into my system. I have an EJB3 entity called Person that is managed by the entity manager. But I also inject a a regular JDBC data-source into my bean that keeps track of login stats. The login_stats table may be a million rows long and denormalized for reporting efficiency so it really doesn't map well to an ORM. The entity manager and the data source could be pointing to the same or different databases, I don't care.

The cool thing is even though I'm using entity beans and JDBC, the application server makes sure they are both enlisted in the same XA transaction. So even though I ask the container for a regular JDBC data-source by the time I get it an XA transaction has already started and is being managed. I could save/update some EB3 entities and execute some JDBC statements and they will be fully ACID compliant.

So yes Matilda, you can have your EJB3 and JDBC too. By using JEE we are not tying ourselves to a predetermined set of abstractions. If I want an ORM, I get that... If I want to do straight JDBC, I get that also. And if I want to do both, I can have that too AND they all play by the same ACID rules.

P.S. You notice that I have to have this function throw a JDBCException. I wish JDBC would follow the path of EJB3 persistence and make their Exception hierarchy extend RuntimeException rather than Exception. But thats just a picky observation, not a framework killer

Active record and those darn caps

2007-09-28T10:31:00.000-07:00

Due to my recent illness I've had a lot of time to myself in doors. So I've been looking a lot of technologies that have been on my watch list. When looking at ruby's active record I can't help but like and hate it at the same time.

One thing Ruby-lovers pride themselves on is "do as little as possible an get as much for free as possible." Take the following active record example:

class Person < ActiveRecord::Base 
  belongs_to :company 
end

class Company < ActiveRecord::Base 
  has_many :people 
end

Gavin King, the creator of Hibernate ORM has an interesting comment about this :

At this point, most developers are thinking um, ok, so how the hell am I supposed to know what attributes a Company has by looking at my code? And how can my IDE auto-complete them? Of course, the Rails folks have a quick answer to this question Oh, just fire up your database client and look in the database!. Then, assuming that you know ActiveRecord's automagic capitalization and pluralization rules perfectly, you will be able to guess the names of the attributes of your own Company class, and type them in manually.

Somehow, excitement about the Ruby language has warped their perceptions to such an extent that these people actually believe that this is a good thing!

I completely agree. Ruby plays some magic and when a record is pulled from the database ruby inspects it and adds the needed fields to these "active record" ruby classes. While this is neat and keeps developers from having to plan out their ORM entities I can't help but see the possible problems with doing this on a very complex schema AND entity model.

crap in the fan

2007-09-27T12:25:00.000-07:00

Obie Fernandez, a Ruby evangelists, created a post saying why java is inferior to anything ruby and ruby on rails. Of course being me I can't pass this up.

http://www.jroller.com/obie/entry/top_10_reasons_why_java

Obviously even a ruby fan can read this and see that Obie really wanted to do little more than spread FUD and get his panties in a bunch (and I do say panties because anyone who wines like that must be wearing pink-lacy panties over his squirrel-sized package).

However, his morning-after post showed that he does have some reasoning within him:

http://www.jroller.com/obie/entry/what_subtelty_and_suck_ass

I've been doing a lot of "battling" lately with ruby evangelists about why I still do a lot of my deployments in JEE. I think he has a good quote here:

"Like Gavin pointed out, you can't do an app with thousands of model classes in Rails. Okay, but I'm really not trying to say that you should try. Right now, I personally have no interest in working on those kinds of monstrosities, but to each his own. "

I completely agree with this. Ruby on Rails is great for small, database driven apps that have the functionality of CRUD. Go beyond that and its simply too much heavy lifting and you need to look into something more heavy-weight, like JEE. Most ruby developers I talk to don't really want to tackle the large enterprise projects, they want to do what Rails was meant to do, small database driven websites.

What this is really about is choosing the best tool for the job. Some projects I would love to do in something like rails. Other projects I think I would spend more time trying to get around rails to solve the problems and I need a framework that can handle those requirements.

Of course after the above post the java community fired back. While I'm obviously biased I sort of feel like this one was much more intelligent.

http://www.javalobby.org/java/forums/t101687.html

JMS.. the fruit rollup of non-transactional resources

2007-09-25T19:59:00.000-07:00

Everyone knows I love JMS. Almost all of my deployments take advantage of it and rightly so. Who couldn't love transactional, durable, scalable asynchronous processing.

One thing JMS is great for is wrapping resources that aren't JTA so they can "participate" in global transactions. Notice the quotes because the correct way of doing this would be to write your own JCA connector, but sometimes we don't have 3 months to read the spec.

A good example is if we want to save some stuff to a database and them write a file. If we were to do the following sudo-code, assume this is in a stateless session bean:



@TransactionalAttribute(REQUIRES_NEW)
public void doSomething()
{
    Person p  = new Person();
    p.setName("Joshua");
    entityManager.persist(p);

    File f = new File("myStuff");
    // write stuff to the file.
}

We actually don't know if our transaction succeeded until after this function has been executed. But wait... we have already written a file... Oops!

Enter JMS. Put this file-write code in a message-driven bean and have the above method pump a message to its destination and the message won't get sent unless the transaction completes.

Now there is one last thing to think about... what if the file write fails. Well, you have 2 options.

1) make sure the mdb rolls-back its transaction so the message will flow into a dlq and do your own compensation.
2) just set the destination to not use a dlq. This means if the message will keep being delivered until it can write a file. Not really ACID compliant but you won't loose any files.

scripting here I come

2007-09-24T14:07:00.000-07:00

So I'm ashamed. I've been putting off learning a new scripting language for way too long. Commence beating me with a rubber duck now.

So why do I even need to learn something like this? I consider myself a pretty good C++ and Java programmer. On top of that I feel like I can fully embellish the heavy-lifting fruits of JEE when it comes to building robust servers. And unlike others in the development world I'm just not turned on by syntactical candy like "no accessor methods required."

The main reason is I do think dynamic scripting languages have a place in my deployments and I like some of the things they are doing in the web-tier world. While the interface-driven language characteristics makes Java a powerful player in the services/backend world it can sometimes be a bit tedious when pumping out web flows. So for the time being my services-tier will always be the heavy-weight JEE server. But I need to learn something smooth to fit on the front.

So what are my options? Well the talk on the town is Ruby. I wrangled with this in my previous posts and I'm simply not a fan. Lets forget for a second that its so slow it makes Java in the early 90's look good. My issue is its almost diabetic on syntactical candy. Who cares if you can do 50 things with 3 command words, I still can't understand it! And the parenthesis rule drives me crazy. Either make the developer use them or don't. Because of this I feel like the Ruby community is full of language evangelists who's only defense of Ruby is
"but its so agile". Remember, a scripting language is suppose to be easier to read/understand than embedded C. And personally I think the Rails framework has some serious functional issues.

Groovy is a Ruby-like language that gets compiled to the JVM. I like it because they took the best of java and ruby and blended them together. I'll keep this in my bag of tricks but I'm not sure this is the complete answer. And lets face it, I can't spend my life in a VM.

Python is a little different for me. My co-worker is a big python guy and I've been listening to what he has had to say. I'm a picture-drawing sudo-code-thinking abstract engineer. If we have a design session and you start throwing byte orders out at me I'll get lost until a dry erase board is brought in. Python seems to really target people like me with their sudo-syntax approach. Plus I like some of the more java-like features they have annotations. Also, compared to Ruby their plugin bank seems to be much more full. And most of all when I ask python people why they like it they can actually tell me things about the language they like.

So Python it is! My next few posts will be how a Java and JEE lover tames the rabid python.

[UPDATE] First note about python.. the main website is python.org, not python.com. Lets hope that doesn't show up in the web logs at work.

Sometimes technology isn't enough

2007-08-07T07:25:00.000-07:00

I'm an enterprise java developer. I love writing scalable servers in JEE land, which usually involve a database back-end. The current server that I'm architecting has been using the Mysql 5.0 database and we have been amazed with the performance. Database purists have been bugging me to switch to Postgres because of its rich feature set. I haven't messed with Postgres since college but I have over 8 years experience with MySQL. Still, I evaluated Postgres and found it to be a very nice database.

So if Postgres is a more mature database with a richer feature set why do so many people still use MySQL? Easy... community and marketing! While at Borders book store I found 2 full shelves on MySQL, and only ONE book on Postgres. If you have the money you can get on the phone to MySQL AS support or even have them come out to your business to train/troubleshoot. I love open source software and both databases can be considered that, but MySQL has done a very good job at creating a service-oriented company behind their offering while Postgres has tried to stay with the purists.

So my final decision was to stay with MySQL, not because it was the better database, but because if something went wrong I'd have more options on how to get things solved.

I love EJB3!!!!

2007-06-07T07:03:00.000-07:00

So for the last 4 years or so a lot of the development I've done has revolved around J2EE. This means using things like EJB, JMS, MBeans, and usually other goodies offered buy JBoss. I've always liked EJBs, especially the way they remove concerns from the developer like threading, locking, synchronization and clustering. However, I have never liked how hard they are to develop and deploy. Things are so much easier in EJB3. Here is a comparison.

In EJB2.X

To deploy a stateless EJB in 2.1 you need to write the bean and make sure you extend SessionBean. Then implement a home, make sure you write the correct method! The create your local and remove interfaces and place the correct methods you want to expose in both EXACTLY as they are in the bean definition. Then you need to write 2 very complex xml documents telling the application server how to deploy and manage your bean. Then package it up into a jar, place it in an ear and deploy.

Now to access the bean you need to create a InitialContext. Then do a lookup on the bean and get the home method. Now use the home method to create an instance of the been itself. Now you can use it.

In EJB3

Write a pojo bean with the business methods you want. Have it extend a local interface and have it override methods in that.

Place the @Stateless annotation on your bean telling the app server this is a stateless bean.
Package it up in an ear and deploy.

To access a bean, if you are in another ejb3 just annotate the member variable with:

@EJB
MySessionBean bean;

And when that bean is created it will also create an instance of your bean and inject it!

Now you get the power of EJB with the simplicity of writing plain old java! I love it! For more information you can go to JBoss or get this really great book.

Happy Coding!

ACM World Programming Competition

2007-03-19T06:42:00.000-07:00

The 2007 ACM World programming competition just wrapped up and they have released the problem set (solve it if you can). This holds a special place in my heart because at Baylor I worked for Dr. Poucher, who is the chair of the ACM contest. While most of the competitors at this level are light years ahead of me, its always fun to look around and see what they solved and how they did it. Enjoy.

Class Factory Design Pattern

2007-03-03T21:09:00.000-08:00

This week I was writing some C++ code and got to use one of my favorite design patters, the "Class Factory" pattern. Lets say I have 2 ways to do something, like send a message. I can either send it in email form or snail mail. Clients will use my code to send messages but the logic stays the same for which method to use: 1) if I have an internet connection send it using email 2) if no internet, send it using snail mail.

Obviously in c++ I should create 3 classes, a base Message sender method and 2 subclasses that encapsulate the logic of either sending mail using internet or snail. But what about the logic for using the correct sending method (code that decides which subclass to create). I could require my clients to implement that logic that means that all over my software will be if-then statements creating my objects.

The other option is to create a class factory that encapsulates the logic of which subclass to create. This is shown below (obviously this is example code)


#ifndef MESSAGESENDER_H_
#define MESSAGESENDER_H_


class MessageSenderFactory;


class MessageSender
{
    protected:
        MessageSender();
    public:
       
        virtual ~MessageSender();
       
        virtual void sendMessage(char* message) = 0;
};


class EmailMessageSender : MessageSender
{
    private:
        EmailMessageSender() {};
    public:
   
        friend class MessageSenderFactory;
       
        virtual ~EmailMessageSender() {};
       
        void sendMessage(char * message)  {} ;
   
};

class SnailMailMessageSender :  MessageSender
{
    private:
        SnailMailMessageSender() {};
    public:
        friend class MessageSenderFactory;
       
        void sendMessage(char * message) {};
};


class MessageSenderFactory
{
private:
    MessageSenderFactory();
   
public:

    /**
     * Creates the correct message sender.
     * 1) if we have a internet connection, create a email sender
     * 2) else create a snailmail sender.
     */
    static MessageSender* createSender()
    {
        if ( HAS_INTERNET)
            return new EmailMessageSender();
        
        return new SnailMailMessageSender();
    }
   
};

#endif /*MESSAGESENDER_H_*/

Notice that I create the constructors in my subclasses private and use the friend construct to only allow the factory to use it. This forces other code to go through the MessageSenderFactory to get new MessageSender objects, thus forcing them to use my logic.

Now, all clients have to do to send a message is:


  MessageSender * sender  = MessageSenderFactory::createSender();
sender->sendMessage(message);

Clean and simple. Enjoy creating class factories!

ORMs are not just about objects

2007-02-26T12:15:00.000-08:00

It seems most of my professional career has been built around using java to pull and put stuff in a database of some sort. Whether we are talking about an eGovernment web interface or a content management system I've done a lot of work with java and databases. The past few years I have grown to use and love Object Relational Mappings (ORMs), specifically Hibernate.

ORMs in the software development world are almost a religion, you either love or hate them. I love them because they take a lot of the "Busy" work out of working with databases. Plus they allow you to stay in an object oriented world rather than moving from java to JDBC, which is much more functional.

Usually when I come across someone who is against ORMs they say "Well it doesn't fit with what we are trying to do" or "Its overkill" or "It adds complexity to a simple interface like SQL". And its true, it can be overkill for some projects and it is another thing to learn and manage. Some people would feel more comfortable pulling fields out of a result set than working with the Hibernate API.

However, I think a lot of those people don't fully appreciate what ORMs give you. Of course the most visible is the java object to sql mapping. But another HUGE benefit is persistent lifecycle. I've seen JDBC code that passes a connection through multiple paths, updating a row here, then updating it again, and so on and so on. From an outsider it becomes almost cryptic what is happening and where. And if you change your schema at anytime, you may not know the implications until the actual code is run.

In ORM land lets say I have a User class, that maps to a table some where. Lets say I do the folowing:


User user = new User("Jon");
session.save(user);
....
user.setAge(18);
session.saveOrUpdate(user);
...
user.setAge(22);
sesion.update(user);
...
user.setIsMaleGender(true);
session.update(user);
...

session.commit();

Now in JDBC land this would end up sending several SQL statements to the database, each being processed, locking rows, etc. The code who set the users's age as 19 may not know that later some other code has set it to 22 unless they queried the db again in the same session. If some logic was later performed with the assumption that the age is 19, that could be a problem.

In Hibernate (or most ORMs) the session would keep track of that user object once it is associated with the session ( after the first session.save()) and keep track of updates and even retrievals. Periodically throughout the lifetime of the session hibernate my flush the user object to the db or it may wait until the end. Doesn't matter, however you do know that throughout that session you are not dealing with stale data.

This is an incredibly efficient, complex and useful function that you get for free just by using an ORM. As your code gets more complex and is re factored a few times, problems that can arise from using stale data that can be very difficult to track down (I should know, I've done it for a while).

So whats the moral? ORM just isn't about objects and abstracting SQL statements. Its about treating persistent data as actual entities in your program. If you think an ORM isn't for your app, I challenge you to look again. The complexity of the added layer may simplify things in other places.

Paradigm shift

2007-02-20T11:06:00.000-08:00

Moving into Objective-C with Cocoa requires a whole paradigm shift when talking about OOP. Rather than objects having methods, they respond to messages. And rather than calling methods, you send messages. The interesting thing is you can send any message you want to an object, the question is they may not respond. Take the following class:


#import 

@interface MyClass : NSObject
{
    NSString * name;
   
    NSMutableArray *items;
}

- (void)setName:(NSString *)aName;

@end

Lets assume I've implemented the rest of this. The following code will compile



MyClass * test = [[MyClass alloc] init];

[test someMethod];

Now, notice that there is not a method called "someMethod" in the class MyClass, however I'm still able to "call" it in my source and compile it. The reason is, I'm not really calling "someMethod", but rather sending a message. When this runs it throws an exception but thats still very different than other languages I'm use to. One cool thing is you can ask an object if it would respond to a message before sent. A model that is very much like inspection in Java.

NSStatusItem

2007-02-11T09:39:00.000-08:00

I've been playing with Cocoa on my MacBook and I really wanted to create a Status Item on the "System Menu bar." Translated, I wanted to create a menu on the top-right of the menu bar that was displayed even when another app was brought to the front. When using Interface Builder I really couldn't see a graphical way to do it. After some searching I figured it out. In one of my controllers, I did the following:


NSStatusBar *bar = [NSStatusBar systemStatusBar];
       NSStatusItem * theItem = [bar statusItemWithLength:NSVariableStatusItemLength];
       [theItem retain];  
       [theItem setTitle:NSLocalizedString(@"Menu Title", @"")];
       [theItem setHighlightMode:YES];
       [theItem setToolTip:@"Menu ToolTip"];
       [theItem setEnabled:YES];
       [theItem sendActionOn:NSLeftMouseDownMask];
       [theItem setTarget:self];
       [theItem setAction:@selector(showInterface:)];
       [theItem setMenu:nil];

And boom, I now have a status item called "Menu Title" on the top right that is always displayed. Awesome!

Love your loops and let them love you

2007-02-09T18:53:00.000-08:00

I'm not a language freek, but I can appreciate an improviment when I see it. One of the new features in Java 5 is their new and improved for loop. Lets say I want to iterator over a Vector of Strings.


Vector list = new Vector();

The old way (pre java 5)


Iterator<> iterator = list.iterator();
      while (iterator.hasNext())
      {
          String s = iterator.next();
          System.out.println(s);
      }

And the new way (Java 5+)


for (String s : list)
      {
          System.out.println(s);
      }

Enjoy!

UPDATE: I just noted that my collection generics was removed by blogger's posting thingy. Weird! Just so everyone knows, I was using Java 5 generics on the vector to strongly type it with Strings.

Java 5 Annotations and You

2007-02-07T13:37:00.000-08:00

I was able to introduce Java 5 annotations to a few people at work today and I think I hit a cord. I've always had a feeling that most people don't really understand the point of annotations, I know it look me a while. One of the big uses of annotations is try remove the restriction on defining your structures based on interface implementation.

Current Practice:

A good example is the swing api. Lets say you have a JButton and you want to receive events when that button is pressed. Current practice is to write an Class that implements EventListener and register your class with the JButton. You then will receive Event objects and you can inspect them and figure out what event occurred and if they occurred on the button you are interested. What happens is now in every EventListener you write you are now duplicating the code to inspect events, and route to the specific business logic.

Throw in Reflection:

To me this is weird, Java is one of the few languages that has inspection and reflection however it is still common practice to force concrete implementation on the users of our apis. What would make more sense is to tell a JButton "When this particular action occurs, call this method on this object." Your controllers then become POJOs with the methods containing specific segments of business logic.

One Step further with Annotations:

Now lets take that a step further and add the concept that I just want to give JButton my object and JButton will figure out what methods to call for what events and when, but still not requiring my controller to implement an interface. This can be done with annotations. Rather than giving the users of my API a set of interfaces they have to extend, I can just give them a set of annotations they can use to annotation their classes and the API will do the rest behind the scenes.

Example:
Lets take this annotation.


import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

enum EventType {BUTTON_PUSHED,MOUSE_OVER}

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface EventCallback {

  EventType value();

}

Here I create an annotation that can decorate a method and takes an enumerated value of either BUTTON_PUSHED or MOUSE_OVER (example actions that could occur on a Button).

Now, lets say I'm writing my own GUI library, I can write a Button class to inspect any objects registering themselves for this annotation and remember them. In my Button class the registerActionListener method looks like this now.


public void registerActionListener(Object object)
    {
        Method[] methods = object.getClass().getMethods();
        for (int i=0;i < methods.length;i++)
        {
            if ( methods[i].isAnnotationPresent(EventCallback.class))
            {
                EventCallback annotation = methods[i].getAnnotation(EventCallback.class);
                EventType eventType = annotation.value();
                if ( eventType == EventType.BUTTON_PUSHED)
                {
                    // remember to call this method on this object if the button is pushed
                }
                else if ( eventType == EventType.MOUSE_OVER)
                {
                    // remember to call this method on this object if the the mouse if over
                }
            }
        }
       
       
    }

Here is where you will really see the advantage. Our Event Listener (or controller) now looks very clean and simple.


public class MyController {

      MyButton button = new MyButton();
   
      public MyController()
      {
          button.registerActionListener(this);
      }
   
      @EventCallback(EventType.BUTTON_PUSHED)
      public void myButtonWasPressed()
      {
          System.out.println("My Button Was Pushed");
      }

      @EventCallback(EventType.MOUSE_OVER)
      public void aMouseIsOverMyButton()
      {
          System.out.println("A mouse if over my button");
      }
   
}

Notice that I can now name my callback methods whatever I want. Also, if I'm not interested in a event type, I don't have to even annotation a method for it. This means I get the methods I want called for only the events I'm interested in.

I really think this is the way all Java APIs are going and for a good thing.

Memory Leak Solved!

2007-02-02T10:48:00.000-08:00

Thanks to Chris for having more follow through than I do. Here is his comment on my last post.

I was curious myself, so I googled "cocoa exception leak", and found this:

"Exceptions are effectively out-of-band return values; in Cocoa, this means the exception object itself is autoreleased."

Full details here:

http://chanson.livejournal.com/126035.html

Memory Leak??

2007-02-02T05:44:00.000-08:00

As I've stated in previous posts, I'm trying to learn Cocoa. I'm amazed that with all my mac experience I really have a very basic knowledge of it. I've been hitting a lot of tutorials and I have one big question. I see variations of the following code a lot

Cup *cup = [[Cup alloc] init];

@try {

    [cup fill];

@catch (NSException *exception) {

    NSLog(@"main: Caught %@: %@", [exception name], [exception  reason]);

@finally {

    [cup release];

Lets assume that [cup fill] throws the exception and its caught here. Isn't the fact that the exception is never released a memory leak? I know I'm being picky but I really see this in a lot of tutorials. I would think that in the catch block I should see [exception release]. Cocoa programmers let me have it.

Valet is here to serve you

2007-01-31T18:41:00.000-08:00

Some of my friends started a company called 94-west. Today they are releasing a new product called Valet. Its an awesome utility for Macs that organizes all your apps, interfaces with Apple's voice command, and has a beautiful heads up display like dashboard. I love it and of you have a Mac you should check it out also.

Valet Product Site

Enterprise Service Bus - Mule

2007-01-30T11:40:00.000-08:00

I love ESB. No, thats not a drug, it stands for Enterprise Service Bus and its a Java Enterprise Component. A lot of development work is taking 2 or more existing things and making them talk together. Moving into the server side arena, this usualy entails spanning multiple machines, importing data, transforming data, and ensuring that data that needs to be imported is never lost.

Now, anyone can write a program that watches a folder for new files and parses them when they arrive. But, add in the following and the problem becomes a little harder:

The server that needs these files may be on a different machine than the folder being watched.
If the server goes down half way through parsing a file, I want to recover when I come back up. Loosing a file is unacceptable.
At any point in time I want to be able to monitor files that are being processed.
Depending on how powerful machines are, I want to control how many files are parsed at a time and when.
More than one component may be interested in the data that these files contain.
Maybe I not only want to be able to watch folders, but maybe also FTP site, email accounts, etc.

Now things are much harder. Just adding in common error handling and recovery can be a pretty difficult task.

Enter the Enterprise Service Bus (ESB). This is a java architecture where you configure endpoints (Hot folders, email accounts, ftp accounts, anything), message flows, and connect them all together. It handles error handling, splicing and duplicating messages, delivery and since it runs in a container threading and resource management is handled for you.

In ESB talk, the example above would be configure a File endpoint to watch a hot folder, a transform to parse the file into relevant data, and another endpoint that talks to the server using whatever transport mechanism fits best. If the message is stopped at any point throughout this process (say, because a server is down) it simply waits until it can move on. Plus, since you are using a well-known API, the leaning curve is greatly redulced and it can be expanded easily.

So go learn about the Enterprise Service Bus architecture. Some good ones are:

To see some code here is an programmatic example using Mule. I will configure an "In" folder and an "Out" folder. When files are dropped in the "In" folder, they are successfully moved through a dummy component (called a pass through component) and to the out folder.


QuickConfigurationBuilder builder = new QuickConfigurationBuilder();
     // this will create an admin agent
builder.createStartedManager(true, "tcp://localhost:60504");

// in endpoint
UMOEndpoint inboundEndpoint = new MuleEndpoint("file://c:/MuleTest/In",true);
// out endpoint
UMOEndpoint outboundEndpoint = new MuleEndpoint(file://c:/MuleTest/Out",false);
 
     // creating a flow using 2 endpoints and a passthrough component.
builder.registerComponent(PassThroughComponent.class.getName(), "My Test Component",
inboundEndpoint, outboundEndpoint, new HashMap());

And thats it. By using these few lines we get a file copier with full error handling and recovery. Plus, you can actually wire multiple Mule ESBs together to route these files from machine to machine to allow scalability and interpolation.

Java Message Service

2007-01-26T12:49:00.000-08:00

I was pushing the Java Message Service (JMS) at work today for a few projects. I think people have a huge misconception about JMS. By using a broker and having entities communicate using messages, you completely decouple your delivery, synchronization and communication mechanizm from your business logic.

If I'm a message producer and I open up a connection to a broker and send a message, my code is no different if that broker is:

on the same machine
in the same VM
On a machine across the internet
really a cluster of brokers performing load balancing and fail over.

This means your code is already scalable even if you don't need it. Also, it means you don't have to worry about guaranteed delivery, persistence, and recovery. Its all done for you. You just write the components that send and receive code. Plus, with some of the great JMS brokers like ActiveMQ, you have lots of options in transport and optimization.

Take this example, here we have my producer


 
/** This is an inVM connection.  This could also be a connection
* to a broker on another machine or a cluster of brokers.  T
* The only thing that would change is this string.
*/
ActiveMQConnectionFactory  factory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");
Connection connection = factory.createQueueConnection();
connection.start();
QueueSession session = connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);

// go ahead and create the queue, if it already exists, we will
// just get the one that is there
Queue queue = session.createQueue("MyQueue");
// Creates a message producer to send messages to the queue
MessageProducer producer = session.createProducer(queue);

// create a generic message, you have several to choose from  
Message message = session.createMessage();
// normally you would want to actually put stuff in the message
  
// send the message
producer.send(message);
  
// send more messages if you want
// cleanup
producer.close();
session.close();
connection.stop();

And here is my reciever


 
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Queue;
import javax.jms.QueueConnection;
import javax.jms.QueueConnectionFactory;
import javax.jms.QueueSession;
import org.apache.activemq.ActiveMQConnectionFactory;

public class MyMessageConsumer implements MessageListener{

 QueueConnectionFactory factory = null;
 QueueConnection connection = null;
 Queue queue = null;
 QueueSession session = null;
 MessageConsumer consumer = null;
 
 /**
  * Opens a connection to the broker, creates a queue session and starts
  * consuming messages.  The nice thing about active MQ
  * that if you are using an inVM broker, you don't actually have to start it.
  * It will be created when the first person opens a connection.
  * @throws Exception
  */
 public void start() throws Exception
 {
  
  /** This is an inVM connection.  This could also be a connection
   * to a broker on another machine or a cluster of brokers.  T
   * The only thing that would change is this string.
   */
  factory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");
  
  connection = factory.createQueueConnection();
 
  session = connection.createQueueSession(false, Session.CLIENT_ACKNOWLEDGE);

  // go ahead and create the queue, if it already exists, we will
  // just get the one that is there
  queue = session.createQueue("MyQueue");

  // create a cosumer and add ourselves as the message listener.
  // the consumer abstracts threading and all
  consumer = session.createConsumer(queue);
  consumer.setMessageListener(this);

  // even though we have created a consumer, messages won't be delivered
  // to us until we start the connetion it belongs to.
  connection.start();
 }

 public void stop() throws Exception
 {
  connection.stop();
  connection.close();
 }
 
 /**
  * Callback methoid from our queue session
  * when a message arrives.
  */
 public void onMessage(Message message) {
  
  try {
   System.out.println("Hey!  I got a message");
   // let the broker know we have the message.  If we didn't do this
                      // the broker would assume something bad happened and redeliver.
   message.acknowledge();
  }
  catch (Exception jms)
  {
   jms.printStackTrace();
  }
 }
}

Pretty simple. My producer opens a connection to the broker and publishes a message to the Queue "MyQueue". My receiver creates a consumer on the queue "MyQueue" and sets itself as the message listener. When it receives a message it prints out a string.

In this example the broker is in the same vm as the producer and consumer. To have a broker running on a different machine or to have either the consumer or producer on different machines or to even have a cluster of brokers requires virtually no change to this code. That means this code is scalable from a single JavaVM to 100 servers clustered. Talk about planning for the future. Also note that I in any of my code I don't have to worry about synchronization, threading, and concurrently. Its all abstracted from my by the broker and the JMS clients.

So why don't people use Message Driven Architecture more often? I think a lot of the reason is the learning curve. It is a new methodology and those are difficult to learn. But once you do it makes life so much easer. So go learn JMS, it will make your life easier.

Objective C and Me

2007-01-25T07:37:00.000-08:00

So part of my new years resolution is to learn more technologies. I am very strong in Java, ask me to do something and I can rip it out with Eclipse and a JVM. C++ is my second strong suite. Even though it doesn't look like professionally I'll be moving out of these technologies for a while, I think its important for me to try new things.

Last night I was playing with Cocoa and Objective-C. Apple has heavily adopted these technologies so I wanted to see why they went this direction. Objective-C is an Object Oriented Language that contains the speed and power seen in C/C++, but also has many of the dynamic features you see in Java and Ruby.

C++ lovers talk about the power and low-level access you get from the language. Whether you are interfacing with a piece of hardware or writing a GUI, it can be done directly in C++. However, you miss out on the dynamic features like Reflection and inspection that Java has. But in Java you are restricted by the fact that you exist in a Virtual Machine and everything you need to access in the operating system either needs to be done using the Java API, or some JNI layer which gets you back to C++.

Objective-C fixes this by the way its classes are defined and how you interact with them. Rather can calling methods, you are really sending them messages and receiving responses. You can inspect if an object will respond to certain messages and change that at runtime (a little scary). They have also changed the way inheritance is done by adding sort of a "Role" attribute.

Another neat thing about Objective-C is that it can link against call C++ APIs natively. This is huge for anyone who has had to build a Java JNI interface to some C++ code they need to access, it isn't pretty.

However with all the dynamic stuff in ObjC you do give up some of the stability and security C++ offers. But with great power also comes great responsibility. All-in-all I really like it and wish it was used more in my professional world. I can think of a lot of situations where my C++ programming tasks would be easier with just simple inspection.

2007-01-22T06:24:00.000-08:00

The past few years most of my programming experience has been in Java with C++ on the side. The past year or so I've had a lot of tech friends telling me I need to checkout Ruby. Since one of my earlier jobs where I inherited some nasty perl and jsp code I've usually tried to run away from scripting languages. After downloading ruby and Ruby Eclipse on my MacBook I have to say I'm very impressed.

Where java is mostly centered around building beautiful frameworks and utilizing inversion of control, Ruby is more about getting it done quickly and let the characteristics of the language handle the readability. Especially the Ruby on Rails framework, which is by far the easiest web framework have seen yet for database driven applications (notice I said database driven apps and not all web apps). I'm not saying Ruby replaces Java, but for some things I would seriously look at it.

Anyways, in an effort to learn more I bought a few books on amazon. First one is pretty much the Ruby Bible, The Pragamatic Guide to Ruby. The second is the associated book for Rails. Hopefully after some nights reading I'll have a better understanding of what I can do. I'm sure there will be more Ruby posts on their way.

Java 6 Released

2006-12-11T11:48:00.000-08:00

I really haven’t used this blog for tech stuff lately, but I really have to rave about the newly announced Java 6 release. This is a huge release for the Java community, not only is it the first release fostered from community input, but it contains some huge advances in services that the JVM offers developers.

Once huge thing is scripting. I’ve used scripting engines such as Groovy and found them very useful in real-world senarios. Especially when you need to provide custom functionality to an out-of-the-box software package why not include a custom script rather than creating a special build for a customer. Now you get all of this for free just by using Java 6, awesome!

Here is the http://www.theserverside.com/news/thread.tss?thread_id=43434

Enjoy!