Valet is here to serve you

Some of my friends started a company called 94-west. Today they are releasing a new product called Valet. Its an awesome utility for Macs that organizes all your apps, interfaces with Apple's voice command, and has a beautiful heads up display like dashboard. I love it and of you have a Mac you should check it out also.

Valet Product Site

Enterprise Service Bus - Mule

I love ESB. No, thats not a drug, it stands for Enterprise Service Bus and its a Java Enterprise Component. A lot of development work is taking 2 or more existing things and making them talk together. Moving into the server side arena, this usualy entails spanning multiple machines, importing data, transforming data, and ensuring that data that needs to be imported is never lost.

Now, anyone can write a program that watches a folder for new files and parses them when they arrive. But, add in the following and the problem becomes a little harder:

1. The server that needs these files may be on a different machine than the folder being watched.
2. If the server goes down half way through parsing a file, I want to recover when I come back up. Loosing a file is unacceptable.
   
3. At any point in time I want to be able to monitor files that are being processed.
4. Depending on how powerful machines are, I want to control how many files are parsed at a time and when.
5. More than one component may be interested in the data that these files contain.
6. Maybe I not only want to be able to watch folders, but maybe also FTP site, email accounts, etc.
   

Now things are much harder. Just adding in common error handling and recovery can be a pretty difficult task.

Enter the Enterprise Service Bus (ESB). This is a java architecture where you configure endpoints (Hot folders, email accounts, ftp accounts, anything), message flows, and connect them all together. It handles error handling, splicing and duplicating messages, delivery and since it runs in a container threading and resource management is handled for you.

In ESB talk, the example above would be configure a File endpoint to watch a hot folder, a transform to parse the file into relevant data, and another endpoint that talks to the server using whatever transport mechanism fits best. If the message is stopped at any point throughout this process (say, because a server is down) it simply waits until it can move on. Plus, since you are using a well-known API, the leaning curve is greatly redulced and it can be expanded easily.

So go learn about the Enterprise Service Bus architecture. Some good ones are:

1. Mule
2. ServiceMix
3. JBoss ESB

To see some code here is an programmatic example using Mule. I will configure an "In" folder and an "Out" folder. When files are dropped in the "In" folder, they are successfully moved through a dummy component (called a pass through component) and to the out folder.

QuickConfigurationBuilder builder = new QuickConfigurationBuilder();
     // this will create an admin agent
builder.createStartedManager(true, "tcp://localhost:60504");

// in endpoint
UMOEndpoint inboundEndpoint = new MuleEndpoint("file://c:/MuleTest/In",true);
// out endpoint
UMOEndpoint outboundEndpoint = new MuleEndpoint(file://c:/MuleTest/Out",false);
 
     // creating a flow using 2 endpoints and a passthrough component.
builder.registerComponent(PassThroughComponent.class.getName(), "My Test Component",
inboundEndpoint, outboundEndpoint, new HashMap());

And thats it. By using these few lines we get a file copier with full error handling and recovery. Plus, you can actually wire multiple Mule ESBs together to route these files from machine to machine to allow scalability and interpolation.

Java Message Service

I was pushing the Java Message Service (JMS) at work today for a few projects. I think people have a huge misconception about JMS. By using a broker and having entities communicate using messages, you completely decouple your delivery, synchronization and communication mechanizm from your business logic.

If I'm a message producer and I open up a connection to a broker and send a message, my code is no different if that broker is:

1. on the same machine
2. in the same VM
3. On a machine across the internet
4. really a cluster of brokers performing load balancing and fail over.

This means your code is already scalable even if you don't need it. Also, it means you don't have to worry about guaranteed delivery, persistence, and recovery. Its all done for you. You just write the components that send and receive code. Plus, with some of the great JMS brokers like ActiveMQ, you have lots of options in transport and optimization.

Take this example, here we have my producer

 

/** This is an inVM connection.  This could also be a connection
* to a broker on another machine or a cluster of brokers.  T
* The only thing that would change is this string.
*/
ActiveMQConnectionFactory  factory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");
Connection connection = factory.createQueueConnection();
connection.start();
QueueSession session = connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);

// go ahead and create the queue, if it already exists, we will
// just get the one that is there
Queue queue = session.createQueue("MyQueue");
// Creates a message producer to send messages to the queue
MessageProducer producer = session.createProducer(queue);

// create a generic message, you have several to choose from  
Message message = session.createMessage();
// normally you would want to actually put stuff in the message
  
// send the message
producer.send(message);
  
// send more messages if you want
// cleanup
producer.close();
session.close();
connection.stop();

And here is my reciever

 

import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Queue;
import javax.jms.QueueConnection;
import javax.jms.QueueConnectionFactory;
import javax.jms.QueueSession;
import org.apache.activemq.ActiveMQConnectionFactory;

public class MyMessageConsumer implements MessageListener{

 QueueConnectionFactory factory = null;
 QueueConnection connection = null;
 Queue queue = null;
 QueueSession session = null;
 MessageConsumer consumer = null;
 
 /**
  * Opens a connection to the broker, creates a queue session and starts
  * consuming messages.  The nice thing about active MQ
  * that if you are using an inVM broker, you don't actually have to start it.
  * It will be created when the first person opens a connection.
  * @throws Exception
  */
 public void start() throws Exception
 {
  
  /** This is an inVM connection.  This could also be a connection
   * to a broker on another machine or a cluster of brokers.  T
   * The only thing that would change is this string.
   */
  factory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");
  
  connection = factory.createQueueConnection();
 
  session = connection.createQueueSession(false, Session.CLIENT_ACKNOWLEDGE);

  // go ahead and create the queue, if it already exists, we will
  // just get the one that is there
  queue = session.createQueue("MyQueue");

  // create a cosumer and add ourselves as the message listener.
  // the consumer abstracts threading and all
  consumer = session.createConsumer(queue);
  consumer.setMessageListener(this);

  // even though we have created a consumer, messages won't be delivered
  // to us until we start the connetion it belongs to.
  connection.start();
 }

 public void stop() throws Exception
 {
  connection.stop();
  connection.close();
 }
 
 /**
  * Callback methoid from our queue session
  * when a message arrives.
  */
 public void onMessage(Message message) {
  
  try {
   System.out.println("Hey!  I got a message");
   // let the broker know we have the message.  If we didn't do this
                      // the broker would assume something bad happened and redeliver.
   message.acknowledge();
  }
  catch (Exception jms)
  {
   jms.printStackTrace();
  }
 }
}

Pretty simple. My producer opens a connection to the broker and publishes a message to the Queue "MyQueue". My receiver creates a consumer on the queue "MyQueue" and sets itself as the message listener. When it receives a message it prints out a string.

In this example the broker is in the same vm as the producer and consumer. To have a broker running on a different machine or to have either the consumer or producer on different machines or to even have a cluster of brokers requires virtually no change to this code. That means this code is scalable from a single JavaVM to 100 servers clustered. Talk about planning for the future. Also note that I in any of my code I don't have to worry about synchronization, threading, and concurrently. Its all abstracted from my by the broker and the JMS clients.

So why don't people use Message Driven Architecture more often? I think a lot of the reason is the learning curve. It is a new methodology and those are difficult to learn. But once you do it makes life so much easer. So go learn JMS, it will make your life easier.

Objective C and Me

So part of my new years resolution is to learn more technologies. I am very strong in Java, ask me to do something and I can rip it out with Eclipse and a JVM. C++ is my second strong suite. Even though it doesn't look like professionally I'll be moving out of these technologies for a while, I think its important for me to try new things.

Last night I was playing with Cocoa and Objective-C. Apple has heavily adopted these technologies so I wanted to see why they went this direction. Objective-C is an Object Oriented Language that contains the speed and power seen in C/C++, but also has many of the dynamic features you see in Java and Ruby.

C++ lovers talk about the power and low-level access you get from the language. Whether you are interfacing with a piece of hardware or writing a GUI, it can be done directly in C++. However, you miss out on the dynamic features like Reflection and inspection that Java has. But in Java you are restricted by the fact that you exist in a Virtual Machine and everything you need to access in the operating system either needs to be done using the Java API, or some JNI layer which gets you back to C++.

Objective-C fixes this by the way its classes are defined and how you interact with them. Rather can calling methods, you are really sending them messages and receiving responses. You can inspect if an object will respond to certain messages and change that at runtime (a little scary). They have also changed the way inheritance is done by adding sort of a "Role" attribute.

Another neat thing about Objective-C is that it can link against call C++ APIs natively. This is huge for anyone who has had to build a Java JNI interface to some C++ code they need to access, it isn't pretty.

However with all the dynamic stuff in ObjC you do give up some of the stability and security C++ offers. But with great power also comes great responsibility. All-in-all I really like it and wish it was used more in my professional world. I can think of a lot of situations where my C++ programming tasks would be easier with just simple inspection.

The past few years most of my programming experience has been in Java with C++ on the side. The past year or so I've had a lot of tech friends telling me I need to checkout Ruby. Since one of my earlier jobs where I inherited some nasty perl and jsp code I've usually tried to run away from scripting languages. After downloading ruby and Ruby Eclipse on my MacBook I have to say I'm very impressed.

Where java is mostly centered around building beautiful frameworks and utilizing inversion of control, Ruby is more about getting it done quickly and let the characteristics of the language handle the readability. Especially the Ruby on Rails framework, which is by far the easiest web framework have seen yet for database driven applications (notice I said database driven apps and not all web apps). I'm not saying Ruby replaces Java, but for some things I would seriously look at it.

Anyways, in an effort to learn more I bought a few books on amazon. First one is pretty much the Ruby Bible, The Pragamatic Guide to Ruby. The second is the associated book for Rails. Hopefully after some nights reading I'll have a better understanding of what I can do. I'm sure there will be more Ruby posts on their way.