Call for Action

I've worked on a few open source projects and I have nothing bad to say about any of them. If there is one shortfall in the open source world its that everyone wants to be the hero. Everyone wants to "start" an open source project. But far less people are willing to join existing projects and take them those last tedious few steps necessary to truly make them useful. These steps usually include bug fixing and documentation. This is why sourceforge has hundreds of thousands of projects that will never be used.

At RubyConf Charles Nutter issued a "Call for Action" asking people to help take things those last steps, both on documentation of the new Ruby runtime AND bug fixing for JRuby. So if you are interesting in doing something new, try and help him out. Its the little things that makes most open source projects great.

Charles Nutter's Call to Action Post

JRuby and Seam

After spending a month or so playing with Ruby and Rails I'm really not happy with Rails.  I build systems that have to scale and Rails has some serious issues when it comes to that.  Its great for simple CRUD apps, but if you need to move outside of that or need to scale its got issues.

I do, however, like JRuby.  I'm also a big fan of jBoss Seam.  It just makes sense to me, use the scalability of EJB3 with state management in web apps.  In JBoss Seam 2.0 you can write your Seam components not only as EJB3, but also in Groovy.  I love EJB3 so that would be my first choice.  But I work with people who really want to use Ruby.  To meet in the middle I've been playing around with integrating JRuby into Seam.  Luckly, Seam is designed well to allow this with some learning curve.  

I'll post more later, but I've got a proof of concept where I can write my view in JSF, and a Seam-managed component in JRuby, which then uses JPA for persistence. 

Dreams of Fuel Cells

I live about an hour from the auto-motive capital of the U.S. (well, the former one at least). The big 3 and the UAW are pretty much house-hold names in Michigan. Most people who know me would testify that I'm not a fan of the U.S. auto industry. From union squabbling to companies who produce non-creative, sub quality products and insist that the reason you should buy them is not because they are better, but because its your patriotic duty.

Even though I rarely achieve this, I always strive to be a creative, forward thinking engineer. I believe that sometimes its not always enough to just solve a problem, but solve it in a fantastic way that yourself and others can learn from. I simply don't see this a lot from the big 3, which is why I drive a Honda.

Tonight I saw a commercial for the new Honda hydrogen fuel-cell car, and I was captured. I felt like I was looking at the iPhone of cars. A company whose industry is as old as some governments refuses to subside into the monotonous productions of U.S. big business and big labor competitors.

I need to do more reading to fully understand what this car is, but it looks like it will hit the road in the summer of 2008. While other companies are just trying to stay a float and squabbling about overtime pay for employee uniform changes, Honda is making history.

http://www.honda.com/fuel-cell/

well-defined api

So from being a java/c++ developer for years I've been taught the importance of exposing well-defined save apis. You expose parts of your api that you think is safe for black-box users to incorporate and abstract/protect the rest. This is usually done by the visibility, security, and overriding features of the language.

For instance, if I have a method that I think its unsafe for any class, including sub-classes to call, I'll define it as private.


Ruby seems to think this isn't important. I can do this:

class Base
def aMethod
puts "A private method";
end
private :aMethod
end

class Derived
public :aMethod
end

In one line the sub class "Derived" was able to change the visibility of the method "aMethod". A little scary when trying to define things like data structure libraries and application frameworks that depend on the validity of certain methods.

Ruby on.... jetty?

So I'm playing around with JRuby, and I think its really cool. The java community has really come a long way and created some great tools for doing enterprise deployments, why should the ruby community spend time backtracking and re-inventing the wheel? The JVM has become a proven execution environment and with an open byte-code standard its trivial to compile to it (well not trivial, but do0-able).

So with JRuby you can get the syntactial sugar of ruby with the heavy lifting of such java services and JMS, hibernate, and even servlet containers. Currently Ruby on Rails uses either WeBrick or Mongrel as their web server. Both are written in ruby and, well, slow. If there is one thing the java community has done a good job of the past few years its build fast and reliable servlet containers (just look at tomcat and jetty).

What I'm wondering is how hard would it be to replace mongrel/WeBrick with jetty in a JRuby on Rails app? Just think of the performance gain with Jetty's NIO engine?

sequel to the rescue

No sooner than I pushed the "post" button on my last post did I find sequel. Its a light-weight ORM in Ruby that looks promising. Does all the association fetching and even has 2nd-level cache support.


http://code.google.com/p/ruby-sequel/

Active Record, the sludge in Ruby

So I've been learning Ruby lately, and being a bug ORM guy I decided to checkout the source for active record and see how it works. There are some very serious issues here. Active record uses the dynamic mixin ability of Ruby to do its work, but that same logic kills itself in performance. To explain myself further I we should compare the active record model with JPA.

JPA
Java persistence architecture delegates all ORM work to the persistence manager, an external library that manages db entities. If I want an entity I ask the persistence manager for it, if I want to save, I do it through the persistence manager. When my app starts up the persistence manager looks at all my entities and my schema and creates the necessary object structure to do its ORM magic. The mapping/querying logic is done once and used by everyone.

ActiveRecord
Active record delegates all ORM work to the entity itself. Entities must know how to query/save themselves and if they are transient or persistent. They do this by mixing in methods to to the ruby-defined entities at runtime and then, when an instance of an entity is instantiated, created the necessary links to do its ORM stuff.

The disadvantage of active record is the function of ORM processes is spread throughout however many entities you have. It becomes hard to do things like locking, transactional caching and even transactional enlistment. If I have a 100 entities, each entity has to be concerned with cache invalidation and managing their relationships.


Until the ruby community either fixes this in active record, or comes out with a JPA/Hibernate -like ORM they will never be in the spotlight. If you talk to any ruby-evangelist they always bring up that Twitter uses RoR. And that is true, but if you read any of their posts to scale Twitter had to remove ActiveRecord from their deployment because it was just too slow.

So I think Ruby is cool but I really don't like active record. Anyone interested in porting hibernate to Ruby???

Netbeans and Ruby

I was working on a Ruby project I'll be revealing later, and I was looking for an IDE. I thought I could get by using VI but after a few years of using Java with Eclipse I've been spoiled. After a little looking I found the Netbeans 6 Beta. They have Ruby package that is fantastic! It contains auto-completion, tool tips, and helpers. Make sure you download the version with the Ruby package installed.

http://dlc.sun.com/netbeans/download/6.0/milestones/latest/


If you are writing Ruby code its well worth the look.

C++ Garbage

So I've been having conversations about smart/auto pointers and c++ with some of my friends. We all love java and other more modern languages that have garbage collection but there are times when you need to move back into C/C++ land, and rightfully so. I'm not one of those "java/ruby/python is the language to replace all other languages." I use the best tool for the job.

So anyways, one of the most painstaking things about C++ is its absence of garbage collection. Sure, you can always just create things on the stack and pass-by-value but that has its own issue (both in performance and logic). Smart and auto pointers help but don't always give the performance that java gc does. One of the nice things about the GC in java is it is not tied to the owning thread. When an object no longer has any reference to it that thread doesn't block while the object is destroyed. The garbage collector runs in the background, even in parallel, collecting objects smartly that can be destroyed and only doing the work it thinks it can do without consuming too many resources. If I suddenly un-reference or move out of scope a million objects the VM won't necessarily destroy all million right then. It will, over time, destroy them when it thinks it won't hurt performance.


So back to C++. Using C++ templates it isn't difficult to do reference counting on your objects. So what we really need to implement is a singleton garbage collector/reference manager that would keep track of all managed objects, and when they are no longer referenced would mark them to be removed, and then remove them when it thinks its best.

Part 1 Singleton Reference Manager
So first we would need to create a singleton C++ class:

class ReferenceManager
{
public:
   static ReferenceManager* Instance();
protected:
   ReferenceManager();
   ReferenceManager(const ReferenceManager&);
   ReferenceManager& operator= (const ReferenceManager&);


};


Now, if you have never done singletons in C++, the way we insure that one and only one gets created/destroyed is using a local static:

ReferenceManager* ReferenceManager::Instance ()
{
 static ReferenceManager inst;
 return &inst;
}

This way a local static instance is created the first time this method is called, and after that the same instance is used. Its also important to note that since inst is local static, it will automatically be destroyed when the application terminates.

References can be obtained like this:

ReferenceManager *p1 = ReferenceManager::Instance();
ReferenceManager *p2 = p1->Instance();
ReferenceManager & ref = * ReferenceManager::Instance();

Part 2. Reference Counting:
In my next post I'll create my own smart pointer to do reference counting and communicate its state with this reference manager. I would do that now but its beautiful outside and the lawn needs mowing.

Avoid locking with JMS

One of the hardest things to do in a high-load system is reporting. Sometimes the amount of data you have to keep track of is too big.

Example:
I have a page that is going to be displayed, and need to :

1. Log every view of the page individually.
2. Keep a running total of everytime a page is viewed.
3. Have a max number of times a page can be viewed, and when that has been reached stop showing the page.

Now lets assume that we need to keep all these values in a database so we can scale-up our app server. For requirements 2 & 3 we need to keep an aggregate table with a column that represents a "counter". Without this table we would have to do an aggregation query on the log table (1) to see how many times this has been viewed, as our data grows this will get slow.

So we create an aggregate table table that contains the page name and a counter of how many times that page has been displayed. Every time that page is viewed we insert into the log table and update the counter in the aggregate table.

Locking is a b$$ch

There is a fundamental flaw with the above solution, if I get 100 concurrent page views, the database will have all those page views "lock" on that row in the aggregation table, this is so the db can provide ACID. This will significantly decrease our ability to scale. You could lower your transaction isolation in your db, but you aggregate value consistency may suffer.

JMS to the rescue

A good solution is to use JMS. When a page request comes in we still insert a row into the log table, but with the same XA transaction we also publish a message onto a durable JMS destination. No locking will occur. Now have a consumer (ether a message-driven-bean or a plain jms message consumer) will consume those messages with a durable subscription and update the aggregate asynchronously. Who cares if they lock, it will be on a completely different thread thats not affecting the performance of serving the page.

Obviously its possible to go over your max show variable (race condition that a page is served before the message consumer has a chance to process and update the aggregate) but when you get to high-yield systems you have to start playing payoffs.

So the moral of the story: JMS can play a big role in high-yield systems with its durability and async processing. Next time you have a locking issue or find you are doing too many things during a request, try moving those things behind a durable JMS topic, it may save you some time!

Python and SQLAlchemy

So if you read this blog its no secret I'm a fan of Hibernate. Not because I'm a Java evangelists but really because I have yet to see many of Hibernates features in other ORMs. Hibernate has gone above and beyond to make an ORM that is not only focused on pojo-based database access but also scalability and durability.

You can tune hibernate to do association fetching and second-level caching that meets your load needs and reduce the amount of bottleneck your database is. You can do projection to limit query context. You can also enlist hibernate in java's JEE global transaction to make it durable and ACID compliant. Many of the other languages and ORMs seem to focus more on developer convenience than these things.

In the python world my boss pointed me to SQLAlchemy. Its a python ORM that is based highly on the design of Hibernate. I like what I see, the association mapping and querying is very similar and powerful.

One thing I see that is missing is the powerful second-level caching concept. Post people's response to this is "Oh, use memcache." Well sure but you loose the acidic properties of your database and ORM. Developers are tasked with first checking in memcache for their entitys, then going to the db, and the same with saving. Both of which don't partipicate in the database transaction they are working in. However, after looking at SQLAlchemy a little more I wonder how hard it would be to build transparent transactional enlistment of it with memcache.

Just think, you decorate your SQLAlchemy object with some sort of marker saying that you want it to participate in second-level caching. Then, along with setting up your persistence context (DB access, etc) you also setup a memcache resource. Now when you access and save your SQLAchemy entities it also pushes and pulls them out of memcache behind the scenes and honors the database ACID transactions. This could do wonders for the scalability of python.

This smells like an open source project to me! Comment with ideas and directions.

JEE isn't just about ORM

I had a conversation with a friend about JEE and how they thought it can be overkill. I should preface by saying I do believe there are situations where JEE is overkill, which I won't go into. His argument is if you don't need to access a database with an ORM then you don't need JEE.

I strongly disagree with this, mostly because I do this all the time. I love Hibernate and EJB3 entity beans, but it doesn't work for all schemas. If you have a highly normalized relational schema where you can take advantage of 2nd-level caching then entity beans is a blessing. But if you are recording stats and doing more OLAP stuff then you may want to stick with straight JDBC.

However, JEE works great with JDBC also. In fact most deployments I've done, including one I'm currently responsible for, use both. Take the following example


public @Stateless class PersonStatsBean implements PersonStats {

@Resource(mappedName="java:/LoginStatsDS")
DataSource loginStats;

@PersistantContent
EntityManager em;


public void recordLogin(int person_id) throws JDBCException
{
Person person = (Person)em.find(Person.class,person_id);
Connection con = null;
try {
con = loginStats.getConnection();
PreparedStatement stmt = con.prepareStatement("insert into login_stats(person_id)
values (?)");
stmt.setParameter(person.getId());
stmt.executeUpdate();
con.close();
}


}



In this example I am saving a row that keeps track of every time a user logs into my system. I have an EJB3 entity called Person that is managed by the entity manager. But I also inject a a regular JDBC data-source into my bean that keeps track of login stats. The login_stats table may be a million rows long and denormalized for reporting efficiency so it really doesn't map well to an ORM. The entity manager and the data source could be pointing to the same or different databases, I don't care.

The cool thing is even though I'm using entity beans and JDBC, the application server makes sure they are both enlisted in the same XA transaction. So even though I ask the container for a regular JDBC data-source by the time I get it an XA transaction has already started and is being managed. I could save/update some EB3 entities and execute some JDBC statements and they will be fully ACID compliant.

So yes Matilda, you can have your EJB3 and JDBC too. By using JEE we are not tying ourselves to a predetermined set of abstractions. If I want an ORM, I get that... If I want to do straight JDBC, I get that also. And if I want to do both, I can have that too AND they all play by the same ACID rules.


P.S. You notice that I have to have this function throw a JDBCException. I wish JDBC would follow the path of EJB3 persistence and make their Exception hierarchy extend RuntimeException rather than Exception. But thats just a picky observation, not a framework killer

Active record and those darn caps

Due to my recent illness I've had a lot of time to myself in doors. So I've been looking a lot of technologies that have been on my watch list. When looking at ruby's active record I can't help but like and hate it at the same time.

One thing Ruby-lovers pride themselves on is "do as little as possible an get as much for free as possible." Take the following active record example:

class Person < ActiveRecord::Base 
  belongs_to :company 
end

class Company < ActiveRecord::Base 
  has_many :people 
end

Gavin King, the creator of Hibernate ORM has an interesting comment about this :

At this point, most developers are thinking um, ok, so how the hell am I supposed to know what attributes a Company has by looking at my code? And how can my IDE auto-complete them? Of course, the Rails folks have a quick answer to this question Oh, just fire up your database client and look in the database!. Then, assuming that you know ActiveRecord's automagic capitalization and pluralization rules perfectly, you will be able to guess the names of the attributes of your own Company class, and type them in manually.

Somehow, excitement about the Ruby language has warped their perceptions to such an extent that these people actually believe that this is a good thing!

I completely agree. Ruby plays some magic and when a record is pulled from the database ruby inspects it and adds the needed fields to these "active record" ruby classes. While this is neat and keeps developers from having to plan out their ORM entities I can't help but see the possible problems with doing this on a very complex schema AND entity model.

crap in the fan

Obie Fernandez, a Ruby evangelists, created a post saying why java is inferior to anything ruby and ruby on rails. Of course being me I can't pass this up.


http://www.jroller.com/obie/entry/top_10_reasons_why_java

Obviously even a ruby fan can read this and see that Obie really wanted to do little more than spread FUD and get his panties in a bunch (and I do say panties because anyone who wines like that must be wearing pink-lacy panties over his squirrel-sized package).

However, his morning-after post showed that he does have some reasoning within him:

http://www.jroller.com/obie/entry/what_subtelty_and_suck_ass


I've been doing a lot of "battling" lately with ruby evangelists about why I still do a lot of my deployments in JEE. I think he has a good quote here:

"Like Gavin pointed out, you can't do an app with thousands of model classes in Rails. Okay, but I'm really not trying to say that you should try. Right now, I personally have no interest in working on those kinds of monstrosities, but to each his own. "


I completely agree with this. Ruby on Rails is great for small, database driven apps that have the functionality of CRUD. Go beyond that and its simply too much heavy lifting and you need to look into something more heavy-weight, like JEE. Most ruby developers I talk to don't really want to tackle the large enterprise projects, they want to do what Rails was meant to do, small database driven websites.

What this is really about is choosing the best tool for the job. Some projects I would love to do in something like rails. Other projects I think I would spend more time trying to get around rails to solve the problems and I need a framework that can handle those requirements.


Of course after the above post the java community fired back. While I'm obviously biased I sort of feel like this one was much more intelligent.

http://www.javalobby.org/java/forums/t101687.html

JMS.. the fruit rollup of non-transactional resources

Everyone knows I love JMS. Almost all of my deployments take advantage of it and rightly so. Who couldn't love transactional, durable, scalable asynchronous processing.

One thing JMS is great for is wrapping resources that aren't JTA so they can "participate" in global transactions. Notice the quotes because the correct way of doing this would be to write your own JCA connector, but sometimes we don't have 3 months to read the spec.

A good example is if we want to save some stuff to a database and them write a file. If we were to do the following sudo-code, assume this is in a stateless session bean:



@TransactionalAttribute(REQUIRES_NEW)
public void doSomething()
{
Person p = new Person();
p.setName("Joshua");
entityManager.persist(p);

File f = new File("myStuff");
// write stuff to the file.
}



We actually don't know if our transaction succeeded until after this function has been executed. But wait... we have already written a file... Oops!

Enter JMS. Put this file-write code in a message-driven bean and have the above method pump a message to its destination and the message won't get sent unless the transaction completes.

Now there is one last thing to think about... what if the file write fails. Well, you have 2 options.

1) make sure the mdb rolls-back its transaction so the message will flow into a dlq and do your own compensation.
2) just set the destination to not use a dlq. This means if the message will keep being delivered until it can write a file. Not really ACID compliant but you won't loose any files.

scripting here I come

So I'm ashamed. I've been putting off learning a new scripting language for way too long. Commence beating me with a rubber duck now.

So why do I even need to learn something like this? I consider myself a pretty good C++ and Java programmer. On top of that I feel like I can fully embellish the heavy-lifting fruits of JEE when it comes to building robust servers. And unlike others in the development world I'm just not turned on by syntactical candy like "no accessor methods required."

The main reason is I do think dynamic scripting languages have a place in my deployments and I like some of the things they are doing in the web-tier world. While the interface-driven language characteristics makes Java a powerful player in the services/backend world it can sometimes be a bit tedious when pumping out web flows. So for the time being my services-tier will always be the heavy-weight JEE server. But I need to learn something smooth to fit on the front.

So what are my options? Well the talk on the town is Ruby. I wrangled with this in my previous posts and I'm simply not a fan. Lets forget for a second that its so slow it makes Java in the early 90's look good. My issue is its almost diabetic on syntactical candy. Who cares if you can do 50 things with 3 command words, I still can't understand it! And the parenthesis rule drives me crazy. Either make the developer use them or don't. Because of this I feel like the Ruby community is full of language evangelists who's only defense of Ruby is
"but its so agile". Remember, a scripting language is suppose to be easier to read/understand than embedded C. And personally I think the Rails framework has some serious functional issues.

Groovy is a Ruby-like language that gets compiled to the JVM. I like it because they took the best of java and ruby and blended them together. I'll keep this in my bag of tricks but I'm not sure this is the complete answer. And lets face it, I can't spend my life in a VM.

Python is a little different for me. My co-worker is a big python guy and I've been listening to what he has had to say. I'm a picture-drawing sudo-code-thinking abstract engineer. If we have a design session and you start throwing byte orders out at me I'll get lost until a dry erase board is brought in. Python seems to really target people like me with their sudo-syntax approach. Plus I like some of the more java-like features they have annotations. Also, compared to Ruby their plugin bank seems to be much more full. And most of all when I ask python people why they like it they can actually tell me things about the language they like.

So Python it is! My next few posts will be how a Java and JEE lover tames the rabid python.


[UPDATE] First note about python.. the main website is python.org, not python.com. Lets hope that doesn't show up in the web logs at work.

Sometimes technology isn't enough

I'm an enterprise java developer.  I love writing scalable servers in JEE land, which usually involve a database back-end.  The current server that I'm architecting has been using the Mysql 5.0 database and we have been amazed with the performance.  Database purists have been bugging me to switch to Postgres because of its rich feature set.  I haven't messed with Postgres since college but I have over 8 years experience with MySQL.  Still, I evaluated Postgres and found it to be a very nice database.  

So if Postgres is a more mature database with a richer feature set why do so many people still use MySQL?  Easy... community and marketing!  While at Borders book store I found 2 full shelves on MySQL, and only ONE book on Postgres.  If you have the money you can get on the phone to MySQL AS support or even have them come out to your business to train/troubleshoot.  I love open source software and both databases can be considered that, but MySQL has done a very good job at creating a service-oriented company behind their offering while Postgres has tried to stay with the purists. 

So my final decision was to stay with MySQL, not because it was the better database, but because if something went wrong I'd have more options on how to get things solved. 

I love EJB3!!!!

So for the last 4 years or so a lot of the development I've done has revolved around J2EE. This means using things like EJB, JMS, MBeans, and usually other goodies offered buy JBoss. I've always liked EJBs, especially the way they remove concerns from the developer like threading, locking, synchronization and clustering. However, I have never liked how hard they are to develop and deploy. Things are so much easier in EJB3. Here is a comparison.

In EJB2.X

To deploy a stateless EJB in 2.1 you need to write the bean and make sure you extend SessionBean. Then implement a home, make sure you write the correct method! The create your local and remove interfaces and place the correct methods you want to expose in both EXACTLY as they are in the bean definition. Then you need to write 2 very complex xml documents telling the application server how to deploy and manage your bean. Then package it up into a jar, place it in an ear and deploy.

Now to access the bean you need to create a InitialContext. Then do a lookup on the bean and get the home method. Now use the home method to create an instance of the been itself. Now you can use it.

In EJB3

Write a pojo bean with the business methods you want. Have it extend a local interface and have it override methods in that.

Place the @Stateless annotation on your bean telling the app server this is a stateless bean.
Package it up in an ear and deploy.

To access a bean, if you are in another ejb3 just annotate the member variable with:

@EJB
MySessionBean bean;

And when that bean is created it will also create an instance of your bean and inject it!

Now you get the power of EJB with the simplicity of writing plain old java! I love it! For more information you can go to JBoss or get this really great book.


Happy Coding!

ACM World Programming Competition

The 2007 ACM World programming competition just wrapped up and they have released the problem set (solve it if you can). This holds a special place in my heart because at Baylor I worked for Dr. Poucher, who is the chair of the ACM contest. While most of the competitors at this level are light years ahead of me, its always fun to look around and see what they solved and how they did it. Enjoy.

Class Factory Design Pattern

This week I was writing some C++ code and got to use one of my favorite design patters, the "Class Factory" pattern. Lets say I have 2 ways to do something, like send a message. I can either send it in email form or snail mail. Clients will use my code to send messages but the logic stays the same for which method to use: 1) if I have an internet connection send it using email 2) if no internet, send it using snail mail.

Obviously in c++ I should create 3 classes, a base Message sender method and 2 subclasses that encapsulate the logic of either sending mail using internet or snail. But what about the logic for using the correct sending method (code that decides which subclass to create). I could require my clients to implement that logic that means that all over my software will be if-then statements creating my objects.

The other option is to create a class factory that encapsulates the logic of which subclass to create. This is shown below (obviously this is example code)

#ifndef MESSAGESENDER_H_
#define MESSAGESENDER_H_


class MessageSenderFactory;


class MessageSender
{
    protected:
        MessageSender();
    public:
       
        virtual ~MessageSender();
       
        virtual void sendMessage(char* message) = 0;
};


class EmailMessageSender : MessageSender
{
    private:
        EmailMessageSender() {};
    public:
   
        friend class MessageSenderFactory;
       
        virtual ~EmailMessageSender() {};
       
        void sendMessage(char * message)  {} ;
   
};

class SnailMailMessageSender :  MessageSender
{
    private:
        SnailMailMessageSender() {};
    public:
        friend class MessageSenderFactory;
       
        void sendMessage(char * message) {};
};


class MessageSenderFactory
{
private:
    MessageSenderFactory();
   
public:

    /**
     * Creates the correct message sender.
     * 1) if we have a internet connection, create a email sender
     * 2) else create a snailmail sender.
     */
    static MessageSender* createSender()
    {
        if ( HAS_INTERNET)
            return new EmailMessageSender();
        
        return new SnailMailMessageSender();
    }
   
};

#endif /*MESSAGESENDER_H_*/

Notice that I create the constructors in my subclasses private and use the friend construct to only allow the factory to use it. This forces other code to go through the MessageSenderFactory to get new MessageSender objects, thus forcing them to use my logic.

Now, all clients have to do to send a message is:

  MessageSender * sender  = MessageSenderFactory::createSender();
sender->sendMessage(message);

Clean and simple. Enjoy creating class factories!

ORMs are not just about objects

It seems most of my professional career has been built around using java to pull and put stuff in a database of some sort. Whether we are talking about an eGovernment web interface or a content management system I've done a lot of work with java and databases. The past few years I have grown to use and love Object Relational Mappings (ORMs), specifically Hibernate.

ORMs in the software development world are almost a religion, you either love or hate them. I love them because they take a lot of the "Busy" work out of working with databases. Plus they allow you to stay in an object oriented world rather than moving from java to JDBC, which is much more functional.

Usually when I come across someone who is against ORMs they say "Well it doesn't fit with what we are trying to do" or "Its overkill" or "It adds complexity to a simple interface like SQL". And its true, it can be overkill for some projects and it is another thing to learn and manage. Some people would feel more comfortable pulling fields out of a result set than working with the Hibernate API.

However, I think a lot of those people don't fully appreciate what ORMs give you. Of course the most visible is the java object to sql mapping. But another HUGE benefit is persistent lifecycle. I've seen JDBC code that passes a connection through multiple paths, updating a row here, then updating it again, and so on and so on. From an outsider it becomes almost cryptic what is happening and where. And if you change your schema at anytime, you may not know the implications until the actual code is run.

In ORM land lets say I have a User class, that maps to a table some where. Lets say I do the folowing:

User user = new User("Jon");
session.save(user);
....
user.setAge(18);
session.saveOrUpdate(user);
...
user.setAge(22);
sesion.update(user);
...
user.setIsMaleGender(true);
session.update(user);
...

session.commit();

Now in JDBC land this would end up sending several SQL statements to the database, each being processed, locking rows, etc. The code who set the users's age as 19 may not know that later some other code has set it to 22 unless they queried the db again in the same session. If some logic was later performed with the assumption that the age is 19, that could be a problem.

In Hibernate (or most ORMs) the session would keep track of that user object once it is associated with the session ( after the first session.save()) and keep track of updates and even retrievals. Periodically throughout the lifetime of the session hibernate my flush the user object to the db or it may wait until the end. Doesn't matter, however you do know that throughout that session you are not dealing with stale data.

This is an incredibly efficient, complex and useful function that you get for free just by using an ORM. As your code gets more complex and is re factored a few times, problems that can arise from using stale data that can be very difficult to track down (I should know, I've done it for a while).

So whats the moral? ORM just isn't about objects and abstracting SQL statements. Its about treating persistent data as actual entities in your program. If you think an ORM isn't for your app, I challenge you to look again. The complexity of the added layer may simplify things in other places.

Paradigm shift

Moving into Objective-C with Cocoa requires a whole paradigm shift when talking about OOP. Rather than objects having methods, they respond to messages. And rather than calling methods, you send messages. The interesting thing is you can send any message you want to an object, the question is they may not respond. Take the following class:

#import 

@interface MyClass : NSObject
{
    NSString * name;
   
    NSMutableArray *items;
}

- (void)setName:(NSString *)aName;

@end

Lets assume I've implemented the rest of this. The following code will compile

MyClass * test = [[MyClass alloc] init];

[test someMethod];

Now, notice that there is not a method called "someMethod" in the class MyClass, however I'm still able to "call" it in my source and compile it. The reason is, I'm not really calling "someMethod", but rather sending a message. When this runs it throws an exception but thats still very different than other languages I'm use to. One cool thing is you can ask an object if it would respond to a message before sent. A model that is very much like inspection in Java.

NSStatusItem

I've been playing with Cocoa on my MacBook and I really wanted to create a Status Item on the "System Menu bar." Translated, I wanted to create a menu on the top-right of the menu bar that was displayed even when another app was brought to the front. When using Interface Builder I really couldn't see a graphical way to do it. After some searching I figured it out. In one of my controllers, I did the following:

NSStatusBar *bar = [NSStatusBar systemStatusBar];
       NSStatusItem * theItem = [bar statusItemWithLength:NSVariableStatusItemLength];
       [theItem retain];  
       [theItem setTitle:NSLocalizedString(@"Menu Title", @"")];
       [theItem setHighlightMode:YES];
       [theItem setToolTip:@"Menu ToolTip"];
       [theItem setEnabled:YES];
       [theItem sendActionOn:NSLeftMouseDownMask];
       [theItem setTarget:self];
       [theItem setAction:@selector(showInterface:)];
       [theItem setMenu:nil];

And boom, I now have a status item called "Menu Title" on the top right that is always displayed. Awesome!

Love your loops and let them love you

I'm not a language freek, but I can appreciate an improviment when I see it. One of the new features in Java 5 is their new and improved for loop. Lets say I want to iterator over a Vector of Strings.

Vector list = new Vector();

The old way (pre java 5)

Iterator<> iterator = list.iterator();
      while (iterator.hasNext())
      {
          String s = iterator.next();
          System.out.println(s);
      }

And the new way (Java 5+)

for (String s : list)
      {
          System.out.println(s);
      }

Enjoy!


UPDATE: I just noted that my collection generics was removed by blogger's posting thingy. Weird! Just so everyone knows, I was using Java 5 generics on the vector to strongly type it with Strings.

Java 5 Annotations and You

I was able to introduce Java 5 annotations to a few people at work today and I think I hit a cord. I've always had a feeling that most people don't really understand the point of annotations, I know it look me a while. One of the big uses of annotations is try remove the restriction on defining your structures based on interface implementation.

Current Practice:

A good example is the swing api. Lets say you have a JButton and you want to receive events when that button is pressed. Current practice is to write an Class that implements EventListener and register your class with the JButton. You then will receive Event objects and you can inspect them and figure out what event occurred and if they occurred on the button you are interested. What happens is now in every EventListener you write you are now duplicating the code to inspect events, and route to the specific business logic.

Throw in Reflection:

To me this is weird, Java is one of the few languages that has inspection and reflection however it is still common practice to force concrete implementation on the users of our apis. What would make more sense is to tell a JButton "When this particular action occurs, call this method on this object." Your controllers then become POJOs with the methods containing specific segments of business logic.

One Step further with Annotations:

Now lets take that a step further and add the concept that I just want to give JButton my object and JButton will figure out what methods to call for what events and when, but still not requiring my controller to implement an interface. This can be done with annotations. Rather than giving the users of my API a set of interfaces they have to extend, I can just give them a set of annotations they can use to annotation their classes and the API will do the rest behind the scenes.

Example:
Lets take this annotation.

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

enum EventType {BUTTON_PUSHED,MOUSE_OVER}

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface EventCallback {

  EventType value();

}

Here I create an annotation that can decorate a method and takes an enumerated value of either BUTTON_PUSHED or MOUSE_OVER (example actions that could occur on a Button).

Now, lets say I'm writing my own GUI library, I can write a Button class to inspect any objects registering themselves for this annotation and remember them. In my Button class the registerActionListener method looks like this now.

public void registerActionListener(Object object)
    {
        Method[] methods = object.getClass().getMethods();
        for (int i=0;i < methods.length;i++)
        {
            if ( methods[i].isAnnotationPresent(EventCallback.class))
            {
                EventCallback annotation = methods[i].getAnnotation(EventCallback.class);
                EventType eventType = annotation.value();
                if ( eventType == EventType.BUTTON_PUSHED)
                {
                    // remember to call this method on this object if the button is pushed
                }
                else if ( eventType == EventType.MOUSE_OVER)
                {
                    // remember to call this method on this object if the the mouse if over
                }
            }
        }
       
       
    }

Here is where you will really see the advantage. Our Event Listener (or controller) now looks very clean and simple.

public class MyController {

      MyButton button = new MyButton();
   
      public MyController()
      {
          button.registerActionListener(this);
      }
   
      @EventCallback(EventType.BUTTON_PUSHED)
      public void myButtonWasPressed()
      {
          System.out.println("My Button Was Pushed");
      }

      @EventCallback(EventType.MOUSE_OVER)
      public void aMouseIsOverMyButton()
      {
          System.out.println("A mouse if over my button");
      }
   
}

Notice that I can now name my callback methods whatever I want. Also, if I'm not interested in a event type, I don't have to even annotation a method for it. This means I get the methods I want called for only the events I'm interested in.

I really think this is the way all Java APIs are going and for a good thing.

Memory Leak Solved!

Thanks to Chris for having more follow through than I do. Here is his comment on my last post.


I was curious myself, so I googled "cocoa exception leak", and found this:

"Exceptions are effectively out-of-band return values; in Cocoa, this means the exception object itself is autoreleased."

Full details here:

http://chanson.livejournal.com/126035.html

Memory Leak??

As I've stated in previous posts, I'm trying to learn Cocoa. I'm amazed that with all my mac experience I really have a very basic knowledge of it. I've been hitting a lot of tutorials and I have one big question. I see variations of the following code a lot

Cup *cup = [[Cup alloc] init];

@try {

[cup fill];

}

@catch (NSException *exception) {

NSLog(@"main: Caught %@: %@", [exception name], [exception reason]);

}

@finally {

[cup release];

}

Lets assume that [cup fill] throws the exception and its caught here. Isn't the fact that the exception is never released a memory leak? I know I'm being picky but I really see this in a lot of tutorials. I would think that in the catch block I should see [exception release]. Cocoa programmers let me have it.

Valet is here to serve you

Some of my friends started a company called 94-west. Today they are releasing a new product called Valet. Its an awesome utility for Macs that organizes all your apps, interfaces with Apple's voice command, and has a beautiful heads up display like dashboard. I love it and of you have a Mac you should check it out also.

Valet Product Site

Enterprise Service Bus - Mule

I love ESB. No, thats not a drug, it stands for Enterprise Service Bus and its a Java Enterprise Component. A lot of development work is taking 2 or more existing things and making them talk together. Moving into the server side arena, this usualy entails spanning multiple machines, importing data, transforming data, and ensuring that data that needs to be imported is never lost.

Now, anyone can write a program that watches a folder for new files and parses them when they arrive. But, add in the following and the problem becomes a little harder:

1. The server that needs these files may be on a different machine than the folder being watched.
2. If the server goes down half way through parsing a file, I want to recover when I come back up. Loosing a file is unacceptable.
   
3. At any point in time I want to be able to monitor files that are being processed.
4. Depending on how powerful machines are, I want to control how many files are parsed at a time and when.
5. More than one component may be interested in the data that these files contain.
6. Maybe I not only want to be able to watch folders, but maybe also FTP site, email accounts, etc.
   

Now things are much harder. Just adding in common error handling and recovery can be a pretty difficult task.

Enter the Enterprise Service Bus (ESB). This is a java architecture where you configure endpoints (Hot folders, email accounts, ftp accounts, anything), message flows, and connect them all together. It handles error handling, splicing and duplicating messages, delivery and since it runs in a container threading and resource management is handled for you.

In ESB talk, the example above would be configure a File endpoint to watch a hot folder, a transform to parse the file into relevant data, and another endpoint that talks to the server using whatever transport mechanism fits best. If the message is stopped at any point throughout this process (say, because a server is down) it simply waits until it can move on. Plus, since you are using a well-known API, the leaning curve is greatly redulced and it can be expanded easily.

So go learn about the Enterprise Service Bus architecture. Some good ones are:

1. Mule
2. ServiceMix
3. JBoss ESB

To see some code here is an programmatic example using Mule. I will configure an "In" folder and an "Out" folder. When files are dropped in the "In" folder, they are successfully moved through a dummy component (called a pass through component) and to the out folder.

QuickConfigurationBuilder builder = new QuickConfigurationBuilder();
     // this will create an admin agent
builder.createStartedManager(true, "tcp://localhost:60504");

// in endpoint
UMOEndpoint inboundEndpoint = new MuleEndpoint("file://c:/MuleTest/In",true);
// out endpoint
UMOEndpoint outboundEndpoint = new MuleEndpoint(file://c:/MuleTest/Out",false);
 
     // creating a flow using 2 endpoints and a passthrough component.
builder.registerComponent(PassThroughComponent.class.getName(), "My Test Component",
inboundEndpoint, outboundEndpoint, new HashMap());

And thats it. By using these few lines we get a file copier with full error handling and recovery. Plus, you can actually wire multiple Mule ESBs together to route these files from machine to machine to allow scalability and interpolation.

Java Message Service

I was pushing the Java Message Service (JMS) at work today for a few projects. I think people have a huge misconception about JMS. By using a broker and having entities communicate using messages, you completely decouple your delivery, synchronization and communication mechanizm from your business logic.

If I'm a message producer and I open up a connection to a broker and send a message, my code is no different if that broker is:

1. on the same machine
2. in the same VM
3. On a machine across the internet
4. really a cluster of brokers performing load balancing and fail over.

This means your code is already scalable even if you don't need it. Also, it means you don't have to worry about guaranteed delivery, persistence, and recovery. Its all done for you. You just write the components that send and receive code. Plus, with some of the great JMS brokers like ActiveMQ, you have lots of options in transport and optimization.

Take this example, here we have my producer

 

/** This is an inVM connection.  This could also be a connection
* to a broker on another machine or a cluster of brokers.  T
* The only thing that would change is this string.
*/
ActiveMQConnectionFactory  factory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");
Connection connection = factory.createQueueConnection();
connection.start();
QueueSession session = connection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);

// go ahead and create the queue, if it already exists, we will
// just get the one that is there
Queue queue = session.createQueue("MyQueue");
// Creates a message producer to send messages to the queue
MessageProducer producer = session.createProducer(queue);

// create a generic message, you have several to choose from  
Message message = session.createMessage();
// normally you would want to actually put stuff in the message
  
// send the message
producer.send(message);
  
// send more messages if you want
// cleanup
producer.close();
session.close();
connection.stop();

And here is my reciever

 

import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Queue;
import javax.jms.QueueConnection;
import javax.jms.QueueConnectionFactory;
import javax.jms.QueueSession;
import org.apache.activemq.ActiveMQConnectionFactory;

public class MyMessageConsumer implements MessageListener{

 QueueConnectionFactory factory = null;
 QueueConnection connection = null;
 Queue queue = null;
 QueueSession session = null;
 MessageConsumer consumer = null;
 
 /**
  * Opens a connection to the broker, creates a queue session and starts
  * consuming messages.  The nice thing about active MQ
  * that if you are using an inVM broker, you don't actually have to start it.
  * It will be created when the first person opens a connection.
  * @throws Exception
  */
 public void start() throws Exception
 {
  
  /** This is an inVM connection.  This could also be a connection
   * to a broker on another machine or a cluster of brokers.  T
   * The only thing that would change is this string.
   */
  factory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");
  
  connection = factory.createQueueConnection();
 
  session = connection.createQueueSession(false, Session.CLIENT_ACKNOWLEDGE);

  // go ahead and create the queue, if it already exists, we will
  // just get the one that is there
  queue = session.createQueue("MyQueue");

  // create a cosumer and add ourselves as the message listener.
  // the consumer abstracts threading and all
  consumer = session.createConsumer(queue);
  consumer.setMessageListener(this);

  // even though we have created a consumer, messages won't be delivered
  // to us until we start the connetion it belongs to.
  connection.start();
 }

 public void stop() throws Exception
 {
  connection.stop();
  connection.close();
 }
 
 /**
  * Callback methoid from our queue session
  * when a message arrives.
  */
 public void onMessage(Message message) {
  
  try {
   System.out.println("Hey!  I got a message");
   // let the broker know we have the message.  If we didn't do this
                      // the broker would assume something bad happened and redeliver.
   message.acknowledge();
  }
  catch (Exception jms)
  {
   jms.printStackTrace();
  }
 }
}

Pretty simple. My producer opens a connection to the broker and publishes a message to the Queue "MyQueue". My receiver creates a consumer on the queue "MyQueue" and sets itself as the message listener. When it receives a message it prints out a string.

In this example the broker is in the same vm as the producer and consumer. To have a broker running on a different machine or to have either the consumer or producer on different machines or to even have a cluster of brokers requires virtually no change to this code. That means this code is scalable from a single JavaVM to 100 servers clustered. Talk about planning for the future. Also note that I in any of my code I don't have to worry about synchronization, threading, and concurrently. Its all abstracted from my by the broker and the JMS clients.

So why don't people use Message Driven Architecture more often? I think a lot of the reason is the learning curve. It is a new methodology and those are difficult to learn. But once you do it makes life so much easer. So go learn JMS, it will make your life easier.

Objective C and Me

So part of my new years resolution is to learn more technologies. I am very strong in Java, ask me to do something and I can rip it out with Eclipse and a JVM. C++ is my second strong suite. Even though it doesn't look like professionally I'll be moving out of these technologies for a while, I think its important for me to try new things.

Last night I was playing with Cocoa and Objective-C. Apple has heavily adopted these technologies so I wanted to see why they went this direction. Objective-C is an Object Oriented Language that contains the speed and power seen in C/C++, but also has many of the dynamic features you see in Java and Ruby.

C++ lovers talk about the power and low-level access you get from the language. Whether you are interfacing with a piece of hardware or writing a GUI, it can be done directly in C++. However, you miss out on the dynamic features like Reflection and inspection that Java has. But in Java you are restricted by the fact that you exist in a Virtual Machine and everything you need to access in the operating system either needs to be done using the Java API, or some JNI layer which gets you back to C++.

Objective-C fixes this by the way its classes are defined and how you interact with them. Rather can calling methods, you are really sending them messages and receiving responses. You can inspect if an object will respond to certain messages and change that at runtime (a little scary). They have also changed the way inheritance is done by adding sort of a "Role" attribute.

Another neat thing about Objective-C is that it can link against call C++ APIs natively. This is huge for anyone who has had to build a Java JNI interface to some C++ code they need to access, it isn't pretty.

However with all the dynamic stuff in ObjC you do give up some of the stability and security C++ offers. But with great power also comes great responsibility. All-in-all I really like it and wish it was used more in my professional world. I can think of a lot of situations where my C++ programming tasks would be easier with just simple inspection.

The past few years most of my programming experience has been in Java with C++ on the side. The past year or so I've had a lot of tech friends telling me I need to checkout Ruby. Since one of my earlier jobs where I inherited some nasty perl and jsp code I've usually tried to run away from scripting languages. After downloading ruby and Ruby Eclipse on my MacBook I have to say I'm very impressed.

Where java is mostly centered around building beautiful frameworks and utilizing inversion of control, Ruby is more about getting it done quickly and let the characteristics of the language handle the readability. Especially the Ruby on Rails framework, which is by far the easiest web framework have seen yet for database driven applications (notice I said database driven apps and not all web apps). I'm not saying Ruby replaces Java, but for some things I would seriously look at it.

Anyways, in an effort to learn more I bought a few books on amazon. First one is pretty much the Ruby Bible, The Pragamatic Guide to Ruby. The second is the associated book for Rails. Hopefully after some nights reading I'll have a better understanding of what I can do. I'm sure there will be more Ruby posts on their way.