Showing posts with label java. Show all posts
Showing posts with label java. Show all posts

9/23/2010

Flexjson meet Android

Flexjson 2.1 now supports running Flexjson on Android. So I thought I'd show a quick example of using Flexjson in an Android application. Hopefully this will spark some ideas about what you can use Flexjson for in your own application. I'm going to start simple creating a quick Android app that pulls recipes from Puppy Recipe, parses it using Flexjson, and displays it in a list. Let's get started.

Recipe Puppy has a very simple REST API, almost too simple, that returns responses in JSON. Recipe puppy allows you to search recipes by the ingredients contained within by using a URL parameter i. Individual ingredients are separated by a comma, and URL encoded. Here is a simple example:

http://www.recipepuppy.com/api/?&i=banana,chicken&p=1

Exciting isn't it? If you click that link you'll see the JSON response. It's a little hard to read like that so here is a simple break down with a little formatting:


{
"title":"Recipe Puppy",
"version":0.1,
"href":"https:\/\/kitty.southfox.me:443\/http\/www.recipepuppy.com\/",
"results":[
{
"title":"Chicken Barbados \r\n\r\n",
"href":"https:\/\/kitty.southfox.me:443\/http\/www.kraftfoods.com\/kf\/recipes\/chicken-barbados-53082.aspx",
"ingredients":"chicken, orange zest, chicken, banana, orange juice, brown sugar, flaked coconut",
"thumbnail":"https:\/\/kitty.southfox.me:443\/http\/img.recipepuppy.com\/602538.jpg"
},
...
]
}


This is pretty straight forward. We have a little header and what we really are interested in results property which is an array of recipe objects. So we'll create two simple Java classes to map those data members. RecipeResponse for the header portion, and Recipe which is the object contained within "results" property.

Here are those objects:


public class RecipeResponse {
public String title;
public Double version;
public String href;
public List<Recipe> results;

public RecipeResponse() {
}
}

public class Recipe {

private String title;
private String href;
private String ingredients;
private String thumbnail;
private Drawable thumbnailDrawable;

public Recipe() {
}

public String getTitle() {
return title;
}

public void setTitle(String title) {
this.title = title.trim();
}

}


In the Recipe object I actually created a Java Bean with getter/setter, but I didn't include most of those methods. I did make a point to show the setter for the title property. Turns out some of the data coming out of recipe puppy contains extra newlines characters in the title. To get rid of those I'm doing a trim() in the setter. Flexjson is smart enough to call the setter method if you have defined it instead of setting values directly into the instance variables. However, if you use public instance variables it will set values directly into those too. This was a fix made in 2.1 with respect to using public instance variables during deserialization process. You'll be happy to know it works now.

So let's jump to the usage of Flexjson in the android code. So we create a RecipeActivity that contains a List to display the recipes. We're going to look at the AsyncTask that loads the data using Flexjson. Here is the full code for that:


new AsyncTask=<String, Integer, List<Recipe>>() {

private final ProgressDialog dialog = new ProgressDialog(RecipeActivity.this);

@Override
protected void onPreExecute() {
dialog.setMessage("Loading Recipes...");
dialog.show();
}

@Override
protected List<Recipe> doInBackground(String... strings) {
try {
return getRecipe( null, 1, "banana", "chicken" );
} catch( IOException ex ) {
Log.e( RECIPES, ex.getMessage(), ex );
return Collections.emptyList();
}
}

@Override
protected void onPostExecute(List<Recipe> results) {
if( dialog.isShowing() ) {
dialog.dismiss();
}
Log.d( RECIPES, "Loading " + results.size() + " Recipes" );
recipes.setList( results );
new ThumbnailLoader( recipes ).execute( recipes.toArray( new Recipe[ recipes.size() ]) );
Log.d( RECIPES, "Loaded " + recipes.size() + " Recipes" );
}

protected List<Recipe> getRecipe( String query, int page, String... ingredients ) throws IOException {
String json = HttpClient.getUrlContent( String.format( "http://www.recipepuppy.com/api/?q=%s&i=%s&p=%d",
query != null ? URLEncoder.encode(query) : "",
ingredients.length > 0 ? URLEncoder.encode(join(ingredients,",")) : "",
page ) );
RecipeResponse response = new JSONDeserializer<RecipeResponse>().deserialize(json, RecipeResponse.class );
return response.results;
}
}.execute();


The method your probably most interested in is getRecipe(). This method formats the URL we're going to load. It then loads that URL and passes the results returned as a JSON block to the JSONDeserializer. JSONDeserializer will take a JSON formatted String and bind that into a Java object. In this example, we're binding into a RecipeResponse object. Here is how that is done:


RecipeResponse response = new JSONDeserializer<RecipeResponse>().deserialize(json, RecipeResponse.class );


A single line of code does that. The deserialize() method performs the deserialization and binding. The first argument is the JSON String, and the second is the top level class we want to bind into. Notice we didn't have to mention anything about Recipe. Flexjson is smart enough to use the data types from the top level object to figure out any other data types contained within. So if you refer to the RecipeResponse.results instance variable you can see the List data type with a generic type. Flexjson will use generics whenever possible to figure out concrete types to instantiate. Of course polymorphism, interfaces, abstract classes, and the like causes issues with this, but we're not going into that right now. See the Flexjson home page to find out more.

You'll notice the RecipeResponse object is returned fully populated with the JSON data, but we're really only interested in response.results so we just return that. It'd be nice if Recipe Puppy returns how many total pages there were in the header (hint, hint) so that it was more interesting. Anyway it is beta. That array is then added to the ListAdapter and displayed on the screen.

Other things Flexjson could be used for is saving state by serializing objects to JSON, and then deserializing when Activities are reconstituted. This can be easier than writing ContentProviders to dump stuff into the database. One of my biggest gripes with Android is how between pages objects can be reliably sent because Intent's require you break everything down to primitives. With Flexjson we can just simply serialize an object put that in the Intent, and then deserialize it on the other side. So no more boilerplate code to flatten your objects.

Here's a simple example serializing our recipes to the disk:


File f = app.getFilesDir();
Writer writer = new BufferedWriter( new FileWriter( new File( f, "recipes.json") ) );
try {
new JSONSerializer().deepSerialize(favorites, writer);
writer.flush();
} finally {
writer.close();
}


Now I know there are people worried about performance, but timing the following code this ran on device in less than 40ms which is within the acceptable bounds for UI performance. If you need more performance you can cache the JSONSerializer/JSONDeserializer instance which optimizes data type mappings so it doesn't recompute those when serializes and deserializes. As always measure, measure, measure.

You've gotten an introduction about how Flexjson can make it easier to work with JSON data with Android.

3/12/2009

Grid Computing: Intro To GridGain Talk is Online

I finally got some time to put up the slides, and source code for my talk I gave at the Devnexus conference in Atlanta. Here is the link to the slides, and the source code is here.

1/07/2009

Fun with Fluent Interfaces and Java

I've written about fluent interfaces before, but I thought I'd share this one I use quite a bit. You never know how much you like something until it's gone. I never thought I really liked Java's InputStream and OutputStream that much until I had to do a lot of streaming work in Actionscript. They have no abstraction for doing stream manipulations. But, let's be honest Java's io first settler's haven't changed much since their introduction. In fact their interfaces have not changed one bit. Sad really because they are so ubiquitous. I find I'm always copying data from one stream the other, dealing with IOExceptions, remembering to close streams, etc. And I got really tired of doing it over and over. What started out as static methods has evolved into a very simple object called ExtendInputStream. Extended as in extending the interface to add more rich functionality rather than the use of inheritance.

The greatest single thing about InputStream and OutputStream is that it's the quintessential example of a decorator. Decorator is one of the foundational software patterns. What I love about decorators is the ability to encapsulate related classes behind a new interface while still retaining interoperability with other decorators.

ExtendedInputStream is a InputStream so it can interact just as plain old InputStream would, but it adds methods like copy, closeQuietly, copyAndClose, and integration with File objects which has always been a pet peeve of mine with InputStream. Let's look at some examples:

Here is copying a file to a directory.


new ExtendedInputStream( someFile ).copyAndClose( dir );


One liner! It's amazing how File object doesn't have these methods already implemented, but then again this approach is much more flexible because we can copy this file to any OutputStream. Here is copying a set of files to a zip.


ZipOutputStream zout = new ZipOutputStream( out );
for( File myfile : files ) {
ZipEntry entry = new ZipEntry( myfile.getName() );
zout.putNextEntry( entry );
new ExtendedInputStream( myFile ).copyAndClose( zout );
}


Five lines of code! Not bad given that 4 of those lines is just to work with ZipOutputStream. Notice how I'm not saving the reference to the ExtendedInputStream here. The copyAndClose() method copies the contents of the file to the OutputStream and closes the InputStream. Closing the OutputStream is your responsibility.

And the more general case of copying an plain old InputStream to any OutputStream.



URLConnection remote = new URL("...").openConnection();
new ExtendedInputStream( new URL("...").openStream() ).copyAndClose( remote.openOutputStream() );


Here is a more advanced version. Say we want to pull down a URL and save it to a file on our local filesystem.


File someDirectory = ...;
new ExtendedInputStream( new URL("...").openStream() ).name( "SavedUrl.txt" ).copyAndClose( someDirectory );


Here we use the optional method name() to set the name of the stream so when we save something to a directory it will use this name as the filename. You could have just as easily done new File( someDirectory, "SaveUrl.txt" ), but it's not always convenient.

You can use a similar pattern for increasing the buffer size used when copying as well.


new ExtendedInputStream( new URL("...").openStream() ).bufferSize( 8096 * 2 ).copyAndClose( someDir );


While I have enjoyed writing this simple class I think I've enjoyed using it more so. I really can't start a new Java project without it now. It's a lot of fun to use. I'd be interested in hearing other features people might want to see added.


package com.wrongnotes.util;

import java.io.*;

public class ExtendedInputStream extends InputStream {

private InputStream delegate;
private String name;
private int bufferSize = 8096;

public ExtendedInputStream( InputStream stream ) {
this( "no_name_file", stream );
}

public ExtendedInputStream(String name, InputStream delegate) {
this.name = name;
this.delegate = delegate;
}

public ExtendedInputStream( File src ) throws FileNotFoundException {
name = src.getName();
delegate = new BufferedInputStream( new FileInputStream( src ) );
}

public int read() throws IOException {
return delegate.read();
}

public int read(byte b[]) throws IOException {
return delegate.read(b);
}

public int read(byte b[], int off, int len) throws IOException {
return delegate.read(b,off,len);
}

public long skip(long n) throws IOException {
return delegate.skip(n);
}

public int available() throws IOException {
return delegate.available();
}

public void close() throws IOException {
delegate.close();
}

public synchronized void mark(int readlimit) {
delegate.mark(readlimit);
}

public synchronized void reset() throws IOException {
delegate.reset();
}

public long copy(File dest) throws IOException {
if( dest.isDirectory() ) {
dest = new File( dest, name );
}
FileOutputStream out = new FileOutputStream(dest);
try {
return copy( out );
} finally {
out.close();
}
}

public long copy(OutputStream out) throws IOException {
long total = 0;
byte[] buffer = new byte[bufferSize];
int len;
while ((len = this.read(buffer)) >= 0) {
out.write(buffer, 0, len);
total += len;
}
out.flush();
return total;
}

public void closeQuietly() {
try {
close();
} catch( IOException ioe ) {
// ignore
}
}

public void copyAndClose( File file ) throws IOException {
try {
copy( file );
} finally {
close();
}
}

public void copyAndClose(OutputStream out) throws IOException {
try {
copy( out );
} finally {
close();
}
}

public ExtendedInputStream bufferSize( int size ) {
bufferSize = size;
return this;
}

public ExtendedInputStream name( String newName ) {
name = newName;
return this;
}
}

4/07/2008

Objects as Functions Part II: A lightweight web app validation utility

Binding and validation is something almost all web frameworks must have a solution for. Some are more elegant than others, but the problem with most of them is that they are tied specifically to the use of that framework. I've yet to see a reusable utility that is framework agnostic for handling this. This is another take on Objects as Functions post I did a while back. Only now I'm applying it to binding and valiation for the web. The results are usable by any Java developer using any framework they want. Creating yet another idea "From Frameworks to Object Oriented Utilities."

I had some code that I had written a while back where I coded the binding and validation by hand. In other words it was just a bunch of if else statement ladders. It resembled something like the following:


setEmail( request.getParameter("email") );

List<String> errors = new ArrayList<String>();

if( isNotSpecified( getEmail() ) ) {
errors.add("Email is missing.");
} else if( isNotSpecified( confirmEmail ) ) {
errors.add("Confirm email is missing.");
} else if( validateEmailFormat( getEmail() ) ) {
errors.add("Email address provided is not a valid.");
} else if( validateEmailFormat( confirmEmail ) ) {
errors.add("Confirm Email address provided is not a valid.");
} else if( !confirmEmail.equals( getEmail() ) ) {
errors.add("Email and Confirm Email did not match.");
}

return errors;


Ok so yikes! I just like jumped back 10 years by writing code like that! But, I did it because I was in a situation where the "architect" hadn't really thought about these problems, and hadn't picked a framework that gave us that ability. So most people weren't doing any validation, and very poor binding. Think Vietnam of web apps here.

So after I wrote this code once I knew I needed something better, but it wasn't until I was about to write it again that I decided to go back and try to refactor out the common code into a utility to make my job easier that I came up with a general solution. If you'll notice in that code above there are some handy instance methods I created in this class to help specifying the validation language. So I started pulling those common methods out into a separate class. I'll spare you the details of the refactoring for another blog post. I'll start with some simple examples:


public List<String> validateAndBind( RequestValidater validater ) {
setFirstName( validater.param("firstName").require().toValue() );
return validater.getErrors();
}


This first example simply validates that the parameter "firstName" was specified in the request, fetches that value, and binds it into the instance object using a setter method. You'll notice there is no reflection taking place here. I'm a huge fan of reflection, but I think you'll see that this is so easy you actually don't need it. Remember that even in reflective frameworks you have to specify the validation rules, and specifying the binding (i.e. making you call the setter method by hand) isn't really where all the hard work is.

Going into detail on what this does. The first step requests the "firstName" parameter from the validater object. Then it calls the require() method on it. This methods checks to see if the parameter is present if not it adds a default error message. The RequestValidater object keeps track of all the errors it encounters while executing the validation rules. Finally toValue() method returns that parameter's value as a String passing it to the object's setter method. If the value isn't present it simply returns null.

In this simple example, if firstName parameter was missing then it would create an error message like: "First Name is missing". Because the parameter used camel case (i.e. firstName) the validater can infer a display name from that by breaking apart the parameter's name on the capital letter boundaries. So "firstName" would become "First Name". You can override this by supplying a second parameter to the param() method. Like:


setFirstName( validater.param("firstName", "First name").require().toValue() );


It will also infer using underscores as well (i.e. "first_name" = "First Name"). It's better to accept the default since it's less work, but realize that you can customize it if you so wish. The second way is to supply an actual error message in the require() method as a parameter. While this might be necessary sometimes, particularly with the matches() method, it's usually best to accept the defaults.

Here's another example that validates and binds a date object.


public List<String> validateBindings( RequestValidater validater ) {
setBirthDate( validater.param("birthDate").require().toDate() );

return validater.getErrors();
}



The key difference in this example is the call to toDate() rather than toValue(). The to*() methods convert strings into other values like integer, dates, etc. These methods will usually end the validation rule methods. You can also pass a default value into the to*() methods to provide a default date, integer, etc. Of course you wouldn't do that with a require() validation rule provided.

Here is a couple more examples:


public List<String> validateBindings( RequestValidater validater ) {
setUsername( validater.param("username").require().between( 5, 30).toValue() );
setEmail( validater.param("email").require().validateEmail().equals( validater.param("confirmEmail") );

return validater.getErrors();
}


This example we see some more methods for performing validations. The between() method is used to validate a parameter's length is between the two values. If not it adds an error message. You can see in the email example. Two methods validateEmail() which makes sure the value conforms to an email address, and the equals method which tests to see if the value matches some other value. In this example you can see how the validater.param("confirmEmail") can be used again to refer to another parameter in the request.

Finally, there's a matches() method for making sure parameters conform to a regular expression. Here is an example of that:


public List<String> validateAndBind( RequestValidater validater ) {

Pattern phoneNumber = Pattern.compile("\\(?\\d\\d\\d\\)?(-|\\s)\\d\\d\\d(-|\\s)\\d\\d\\d\\d");

setPhonNumber( validater
.param("phoneNumber")
.require().matches( phoneNumber, "Phone Number provided does not look like a phone number.")
.toValue() );

return validater.getErrors();
}



Here's how you can use the RequestValidater in your controllers:


MyObject obj = new MyObject();
List<Errors> errors = obj.validateBindings( new RequestValidater( request ) );
if( errors.isEmpty() ) {
// no errors means the user's request was valid
} else {
// we have some errors so send them back with the form data.
}



After I finished writing this utility I went back and refactored by code to use it. I had something like 100-150 lines of validation code that I reduced to a simple 8 lines of code. Actually I added some new lines and formatting between some of the chained method calls which inflated it to like 25 lines, but still that's an amazing amount of code reduction. And, that doesn't include the lines I would've written to do validation in the second object.

This is yet another example of how you can use Objects in Java as functions to really change how you can reuse code. Notice that I didn't create some static method utility class to do this because there was state being kept and managed inside RequestValidater for me. If I had a static method I'd have to keep track of the state for me. Even the error messages can be standardized across my entire app by using this class. Also notice that the fact I'm using binding and validating against HttpServletRequest is hidden from my model objects. This is another great example of how encapsulation hides the details of the system from my model objects. Something static utilties can't do for me. Why is this important? Well I didn't go into it, but I also made changes to the RequestValidater such that it's easy to use in unit tests by just instantiating it with a HashMap of parameters. That makes it really easy to automate your validation testing because your model object's aren't bound to the HttpServletRequest interface. Without using encapsulation it wouldn't have been that easy to reuse RequestValidater in a different context. You can see an example in the main method included in the source of how to reuse it in unit tests.

Finally, my last thought on this is that it's a single class. There is no framework you have to adopt to use this code. Java has many choices when it comes to web frameworks. It's both a blessing and a curse, but the reality of the matter is most people are using Struts! Yuck! Why continue to build code that no one else can use? This marks the time when we need to move away from frameworks and to utilities.

Now here's the source:


package com.cci.web.validation;

import javax.servlet.http.HttpServletRequest;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.regex.Pattern;

public class RequestValidater {
private HttpServletRequest request;
private Map<String,String> params;
private List<String> errors = new ArrayList<String>();

public RequestValidater(HttpServletRequest request) {
this.request = request;
}

public RequestValidater( Map<String,String> params ) {
this.params = params;
}

public boolean hasErrors() {
return !errors.isEmpty();
}

public List<String> getErrors() {
return errors;
}

public Parameter param( String name ) {
return new Parameter( name );
}

public Parameter param( String name, String displayName ) {
return new Parameter( name, displayName );
}

protected String getParameter( String name ) {
if( request != null ) {
return request.getParameter(name);
} else {
return params.get(name);
}
}

public class Parameter {
private String displayName;
private String name;
private String value;

public Parameter(String name) {
this.name = name;
this.displayName = convertToDisplay( name );
this.value = getParameter(name);
}

public Parameter(String name, String displayName) {
this.name = name;
this.displayName = displayName;
this.value = getParameter(name);
}

private String convertToDisplay(String camelCase) {
StringBuilder builder = new StringBuilder();
if( !camelCase.contains("_") ) {
builder.append( Character.toTitleCase( camelCase.charAt(0) ) );
for( int i = 1; i < camelCase.length(); i++ ) {
char next = camelCase.charAt(i);

if(Character.isUpperCase( next ) ) {
builder.append( ' ' );
}
builder.append( next );
}
} else {
String[] words = camelCase.split("_");
for( String word : words ) {
builder.append( Character.toUpperCase( word.charAt(0) ) );
builder.append( word.subSequence( 1, word.length() ) );
}
}
return builder.toString();
}

public Parameter require() {
return require( displayName + " is missing." );
}

public Parameter require(String error ) {
if( name == null || name.length() < 1 ) {
errors.add( error );
}
return this;
}

public Parameter between( int minSize, int maxSize ) {
return between( minSize, maxSize, displayName + " must be at least " + minSize + " characters, but no more than " + maxSize + " characters.");
}

public Parameter between( int minSize, int maxSize, String error ) {
if( value == null ) return this;

if( value.length() < minSize || value.length() > maxSize ) {
errors.add( error );
}
return this;
}

public Parameter matches( Pattern pattern, String error ) {
if( value == null ) return this;

if( !pattern.matcher( value ).matches() ) {
errors.add( error );
}
return this;
}

public Parameter validateAsEmail() {
if( value == null ) return this;

if( !value.matches("(\\w|\\.)+@\\w+\\.\\w+(\\.\\w+)*") ) {
errors.add( value + " is not a valid email address.");
}
return this;
}

public Parameter equals( Parameter param ) {
return equals( param.value, displayName + " does not match " + param.displayName + "." );
}

public Parameter equals( String aValue, String error ) {
if( value == null ) return this;

if( !value.equals( aValue ) ) {
errors.add( error );
}
return this;
}

public Date toDate() {
return toDate( "MM/dd/yyyy");
}

public Date toDate( String datePattern ) {
return toDate( datePattern, value + " is not a valid date. (" + datePattern + ")" );
}

public Date toDate( String datePattern, String error ) {
if( value == null ) return null;

try {
SimpleDateFormat dateFormat = new SimpleDateFormat( datePattern );
return dateFormat.parse( value );
} catch( ParseException pex ) {
errors.add( error );
return null;
}
}

public Integer toInt() {
return toInt( (Integer)null );
}

public Integer toInt( Integer defaultVal ) {
return toInt( displayName + " must be a number without a decimal point.", defaultVal );
}

public Integer toInt( String error ) {
return toInt( error, null );
}

public Integer toInt( String error, Integer defaultValue ) {
if( value == null ) return defaultValue;

try {
return Integer.parseInt( value );
} catch( NumberFormatException nex ) {
errors.add( error );
return null;
}
}

public String toValue() {
return value;
}

public String toValue( String defaultValue ) {
return value != null ? value : defaultValue;
}
}

public static void main(String[] args) {
Map<String,String> params = new HashMap<String,String>();
params.put("email", "jep1957@mindspring.com" );
params.put("email1", "this.email@bad" );
params.put("email2", "my address@bad.com" );

validateThese( "jep1957@mindspring.com", "charlie.hubbard@coreconcept.com", "this.email@bad", "my address@bad.com", "bad@bad@bad@bad", "hiya", "foo.bar" );
}

private static void validateThese( String... emails ) {
for( String email : emails ) {
Map<String,String> params = new HashMap<String,String>();
params.put("email", email );

RequestValidater validater = new RequestValidater( params );
String val = validater.param("email").validateAsEmail().toValue();
System.out.println( val + " was " + ( validater.hasErrors() ? "not valid!" : "valid" ) );
}
}
}

2/02/2008

Life without Getters and Setters

Over at Stephan's Blog he mentioned an idea that I had the other day as well. Back when Java Beans came around the getter and setter notation was a convention to help frameworks determine what the properties of an object were. But, now that we have annotations why, other than legacy reasons, would we continue to use them? Just annotate the method with @Get, @Set, or @Property.

The cool thing is that if legacy frameworks used java.bean.Inspector class, you could encapsulate the new fangled annotation style properties inside it, and old frameworks wouldn't need to know that it changed.

This would get us closer to something I'd like in Java 7. Uniform Access Principle. No more would we need to write getter and setters. It's actually a feature that wouldn't harm the language, and you could turn it off with annotations if you had a need. The cool thing about it is that it's completely compiler driven. No language changes. Maybe if we can't get agreement on closures how about this for Java 7?

1/31/2008

Objects as Functions

I've been wanting to write this post for sometime now. It just takes my so long to write a post, usually because of their length, that I don't do it as frequently as I should. I promised more articles in my last post so here we go.

Over the last couple of years I've been learning other languages particularly Javascript and Ruby. I really like Javascript as a language, API could use some help. I love Ruby's style of meta-programming and the fact that everything is an object. What I'm really jealous is how powerful a single function in these languages can be. For example in javascript it's easy to create widgets with Ext.js:


var button = new Ext.Button( 'okButton', {
text: "OK",
click: function( evt ) {
alert("Hey!");
},
"class": "myCssClass"
} );


In Swing this would be 25 lines of setter methods to customize components (OK maybe I'm exaggerating a little). Or in Ruby active record is just so simple and elegant with things like:


books = Book.find( :all, :condition => [ "where pubdate > ?", pubDate ], :include => [ :tags, :comments ] )


The ability to hand in optional arguments regardless of position turns simple functions into really expressive mini languages. Dare I say DSLs. I really miss these features when I have to use Java. Since Java opted for the C++ route of data structures. Data structures live in the API, and not the language, it's difficult for us to mimic this type of power.

I remember reading the Javascript Bible several years ago, and in there is a chapter titled "Functions as Data". And, the author is trying to show how functions can be assigned to variables, and passed around the system such one part of the program is unaware of which exact function it's calling. He's trying break the stereotype of functions being just actions, and show you how functions can be data as well. It's a good chapter for any developer.

This got me thinking about stereotypes we have about other languages. In Java we, all too often, get stuck thinking of our objects as largely data with some marginal functions mashed in to manipulate or manage that data. We usually create huge single instance, infinite lifetime objects that then do all the heavy lifting. However, what if we created objects that lived really only for a small period of time just to perform small functions? In fact what if we took what we would normally write as a function and turned it into an object. How would that change what we could express?

Now we need a problem to practice our idea on. Let's say we want to build a simple email utility because I think everyone who has worked on a web application in the past has had to send out email. For example, say we want to send an email from the site when people register to welcome them to the system, or when they forget their password. And, as you know Javamail, while a pretty good general purpose API, is very verbose to use. So let's say we create some utility method for sending email:


public MailHelper {
public static MimeMessage createEmail( String to, String from, String subject, String template ) {
}
}


The idea of this method is that we give it the to, from, subject, and a template file to use for the body, and it returns a MimeMessage, which is a Javamail class. It's basically an object representing an email. The template is either a Velocity or Freemarker template. I'm going to use Velocity in this example. If you're not familiar with these libraries they're really simple. They take in text like:


Hi $name,

Welcome to my cool new website. Thanks for taking the time to register with us. We're so glad to call you a member. Here is your password: $password in case you forgot, and here are some great places to get started $interestingPlace.

Thanks

The Man


And replace all of the "$" references with something you assign to them. So in your program you might say "name"="Charlie", and "password"="wouldnt you like to know", and "interestingPlace"="Tahiti". If you've used properties in Ant it's really a similar concept. It's not rocket science.

Now back to our utility. The problem with our utility is that we need a way to set all of those interesting variables that we'll substitute into our template. But, how do we do that with our single static method?

In Javascript or Ruby they might just pass a hash map like:


function createEmail( to, from, subject, template, bindings ) {
bindings = bindings || {}; // these notation is used for optional arguments.
}


What's cool about the function above is that if you don't have any bindings, just don't include it. Javascript doesn't require you pass all of the arguments to the function so the first line just happily sets bindings to empty object if they aren't specified.

So if we copy this in Java we'd add a HashMap onto our arguments:


public MailHelper {
public static MimeMessage createEmail( String to, String from, String subject, String template, Map bindings ) {
}
}


Then our usage would be something like:


public static void main( String args[] ) {

Map bindings = new HashMap();
bindings.put("name", "Unsuspecting Luser");
bindings.put("password", "wouldnt you like to know" );
bindings.put("interestingPlaces", "Tahiti" );

MimeMessage message = MailHelper.createEmail( "ycnangerp1@hotmail.com", "kevben@coldmail.ca", "Hello New Member", "email.vm", bindings );
}


Now I don't know about you, but that kinda code drives me nuts. It's so verbose, and
boring! The Javascript and Ruby versions of this are so much more fun. Now what if we wanted to add some attachments, CC some people, or add HTML and text versions of the email? Our poor little static function just can't hang. What we need is something better.

If you're still not convinced that statics aren't up for this consider that I'm leaving out some very important architecture here. Particularly, I need a VelocityEngine instance inside the static function in order to evaluate the template. Now I have to make a choice. One I make a simplifying assumption that I only need one instance of VelcoityEngine and initialize it in the static utility. That would mean I give up on the ability to have multiple template locations, or mail servers, etc. This would make it hard to unit test. It also makes reusing this utility harder because everything is hardcoded structurally. And, singleton's don't help this! The other choice is force the user to manage of the data this static utility needs, but I think that would compromise the ease and usefulness of the utility. Kinda of like well geez if you're going to make me do all of that I might as well just do it myself. Both of them get us into ugly Java.

What if we take our function and turn it into an object? I'm going to approach this from the outside interface, and then show the implementation:


public static void main( String[] args ) {

Emailer emailer = new Emailer();

MimeMessage mime = emailer.email("ycnangerp1@hotmail.com", "Hello New Member", "email.vm" )
.bind( "name", "Bad Spammer" )
.bind("password", "wouldnt you like to know")
.bind("interestingPlaces", "Hell" )
.toMail();
}



Look at that! Much easier to use. Notice how I don't have to create a seperate HashMap. Our new object provides this for us so binding is completely optional. If the email you're sending doesn't have any bindings then just don't call the bind() method. Also notice that emailer instance has a email() method to create an email. It almost reads like English. It also scales nicely for other types of data like attachments, or CC'ing:


public static void main( String[] args ) {

Emailer emailer = new Emailer();

MimeMessage mime = emailer.email("ycnangerp1@hotmail.com", "Hello New Member", "email.vm" )
.bind( "name", "Unsuspecting Luser" )
.bind("password", "wouldnt you like to know")
.bind("interestingPlaces", "Tahiti" )
.cc( "manager@spambot.com" )
.attach( stockHypeImage )
.toMail();
}



I'm using email addresses of known spammers to increase the chances spam bots will pick them up and blast emails at them.

If you'll notice I moved one parameter out from the first function: the from parameter. I'll add this back in at a later point in time. You'll also notice the toMail() method. Since we're making multiple calls we'll need a function to end the transaction or kick off the action. I'm going to enhance the version of the code below to also send the email instead of just creating it. I'm not going to implement the toMail() method since sending is the ultimate goal. So now let's look at the code for implementing this kinda class:


public class Emailer {

private static final Logger logger = Logger.getLogger(Emailer.class);

private String from;
private String url;
private Properties mailProperties;
private VelocityEngine engine;

public Emailer( VelocityEngine anEngine, Properties mailProps, String from, String url ) {
this.engine = anEngine;
this.mailProperties = mailProps;
this.from = from;
this.url = url;
}

public Email email( String to, String subject, String mailTemplate ) {
return new Email( to, subject, mailTemplate );
}

public Email email( String to, String subject, String mailTemplate, String htmlTemplate ) {
return new Email( to, subject, mailTemplate, htmlTemplate );
}

public class Email {
private String to;
private String subject;
private String textTemplate;
private String htmlTemplate;
private Map params;

private Email() {
this.params = new HashMap();
if( url != null ) {
this.params.put("website", url);
}
}

public Email(String to, String subject, String textTemplate) {
this();
this.to = to;
this.subject = subject;
this.textTemplate = textTemplate;
}

public Email(String to, String subject, String textTemplate, String htmlTemplate) {
this();
this.to = to;
this.subject = subject;
this.textTemplate = textTemplate;
this.htmlTemplate = htmlTemplate;
}

public Email bind( String key, Object obj ) {
params.put( key, obj );
return this;
}

public void send() {
try {
Session session = Session.getInstance( mailProperties );
MimeMessage message = new MimeMessage( session );

message.setRecipient( MimeMessage.RecipientType.TO, new InternetAddress( to ) );
message.setFrom( new InternetAddress( from ) );
message.setSubject( subject );

if( htmlTemplate == null ) {
message.setText( renderTemplate( textTemplate ) );
} else {
String text = renderTemplate( textTemplate );
String html = renderTemplate( htmlTemplate );

MimeMultipart part = new MimeMultipart("alternative");
part.addBodyPart( createBody( text,"text/plain") );
part.addBodyPart( createBody( html,"text/html") );

message.setContent( part );
}
message.saveChanges();

Transport transport = session.getTransport("smtp");
transport.connect();
transport.sendMessage( message, message.getAllRecipients() );
transport.close();
} catch( Exception mex ) {
logger.error( "There was an problem emailing: " + subject + " to: " + to, mex );
}
}

private BodyPart createBody(String text, String mimetype) throws MessagingException {
MimeBodyPart body = new MimeBodyPart();
body.setContent( text, mimetype );
return body;
}

private String renderTemplate( String template ) throws Exception {
StringWriter writer = new StringWriter();

Template tmpl = engine.getTemplate( template );
VelocityContext context = new VelocityContext( params );
tmpl.merge( context, writer );
return writer.toString();
}
}
}


The two interesting methods are the Emailer.email() methods. The two versions are for sending text only emails, or multipart HTML emails. At the start it's really quite simple. Invoking email() creates an Email instance which is an inner class for the Emailer. At that point you can call bind() method for every object you want to bind into the template. Notice how bind() returns the Email instance allowing you to chain method calls. Finally once you're done call send() to actually send the email. This was something that you couldn't do with the static method. For one it was too complex to configure through a single method call, and you're static method would have to have access to java properties, velocity engine, etc. By limiting our interaction to statics we force our users to take on more responsibilities. In the static case the user has to use some other utility to send the email. Either we have to force user to manage the architecture by gathering up the velocity engine instance, java mail properties, etc. Or we hard code it with statics in such that our tool will only work for a single velocity engine, from, and java mail properties. Neither is very helpful. One doesn't do enough to make it helpful, and the other limits our architecture and impacts how we can reuse it.

The Emailer class is also split into two parts. The outer class which holds all of the architectural/configuration items (velocity engine, java mail properties, etc). By splitting the code this way those portions can be configured at some place else in the code, say at application initialization, and the parts of the code that need to send email can be blissfully unaware of the architecture used to do so. This outer class acts as the glue between the java mail settings, velocity engine, and simple data like the email address of the author. It's really similar to functional language technique of currying functions.

The second part is the inner class Email. Using an inner classes allows us to create an instance that holds temporary state (i.e. bindings, the template for a specific email, to, subject, etc), but still has access to long living state (i.e. mail host, properties, velocity engine, etc). If we mixed the two inside the Emailer class we couldn't share that across multiple threads. Notice there's no setter methods on Emailer. That means once the Emailer is created other threads can't modify the internal state of it. It's immutable so it's safe to share the Emailer across multiple threads. This pattern allows Emailer to be shared, while Email instances only live on the stack of each Thread. There by separating data being written to (i.e. specifics for sending a single email) from the data that's not changing (i.e. the architecture of sending emails). Safety without synchronizing!

Notice how what started out as our verb function, createEmail(), and turned into a Noun, Email, that acted like a verb. That's usually what happens when you move from static utility methods to active function Object. For lack of a better term.

What's cool about this approach is our Emailer class acts more like a complex function like we saw in Javascript and Ruby. In a single line we easily send an email that requires potentially complex optional data. In addition it allows clients of this class to focus only on sending email, and not worry about the details of how that email is actually sent. Since we're in Java our function Object can tell the user all the options you can call on it with our IDE's code completion feature. Something Ruby and Javascript can't do reliably enough to be useful. With those scripting languages you're relying on the documentation to tell you the specific options of the function.

This pattern in Java looks more like Smalltalk to me. Your functions can end up looking very english-like. By using method chaining and inner classes we can get the same power without having to sacrifice the compact, expressiveness of functions we enjoyed with Javascript and Ruby. Not only did we improve the code's reusability, but we delivered more features by switching from static method to function object. We went from just being able to create the email to actually creating and sending it using objects as functions. In the end, we can now create very powerful single line "functions" just like those scripting languages by using the one construct Java does well. Objects.

Now as I promised da code. This example create a sub directory called templates under your current working directory, and put a test.vm template in there. Copy the template text above, and it will work:


package mail;

import org.apache.log4j.Logger;
import org.apache.velocity.app.VelocityEngine;
import org.apache.velocity.VelocityContext;
import org.apache.velocity.Template;

import javax.mail.*;
import javax.mail.internet.MimeBodyPart;
import javax.mail.internet.MimeMessage;
import javax.mail.internet.MimeMultipart;
import javax.mail.internet.InternetAddress;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.io.StringWriter;

public class Emailer {

private static final Logger logger = Logger.getLogger(Emailer.class);

private String from;
private Properties mailProperties;
private VelocityEngine engine;

protected Emailer() {
}

public Emailer(VelocityEngine engine, Properties mailProperties, String from) {
this.engine = engine;
this.mailProperties = mailProperties;
this.from = from;
}

public Email email( String to, String subject, String mailTemplate ) {
return new Email( to, subject, mailTemplate );
}

public Email email( String to, String subject, String mailTemplate, String htmlTemplate ) {
return new Email( to, subject, mailTemplate, htmlTemplate );
}

public class Email {
private String to;
private String subject;
private String textTemplate;
private String htmlTemplate;
private Map params;

private Email() {
this.params = new HashMap();
}

public Email(String to, String subject, String textTemplate) {
this();
this.to = to;
this.subject = subject;
this.textTemplate = textTemplate;
}

public Email(String to, String subject, String textTemplate, String htmlTemplate) {
this();
this.to = to;
this.subject = subject;
this.textTemplate = textTemplate;
this.htmlTemplate = htmlTemplate;
}

public Email bind( String key, Object obj ) {
params.put( key, obj );
return this;
}

public void send() {
try {
Session session = Session.getInstance( mailProperties );
MimeMessage message = new MimeMessage( session );

message.setRecipient( MimeMessage.RecipientType.TO, new InternetAddress( to ) );
message.setFrom( new InternetAddress( from ) );
message.setSubject( subject );

if( htmlTemplate == null ) {
message.setText( renderTemplate( textTemplate ) );
} else {
String text = renderTemplate( textTemplate );
String html = renderTemplate( htmlTemplate );

MimeMultipart part = new MimeMultipart("alternative");
part.addBodyPart( createBody( text,"text/plain") );
part.addBodyPart( createBody( html,"text/html") );

message.setContent( part );
}
message.saveChanges();

Transport transport = session.getTransport("smtp");
transport.connect();
transport.sendMessage( message, message.getAllRecipients() );
transport.close();
} catch( Exception mex ) {
logger.error( "There was an problem emailing: " + subject + " to: " + to, mex );
}
}

private BodyPart createBody(String text, String mimetype) throws MessagingException {
MimeBodyPart body = new MimeBodyPart();
body.setContent( text, mimetype );
return body;
}

private String renderTemplate( String template ) throws Exception {
StringWriter writer = new StringWriter();

Template tmpl = engine.getTemplate( template );
VelocityContext context = new VelocityContext( params );
tmpl.merge( context, writer );
return writer.toString();
}
}

public static void main(String[] args) throws Exception {
Properties p = new Properties();
p.setProperty("file.resource.loader.path", "./templates");

VelocityEngine engine = new VelocityEngine( p );
engine.init();

String from = "author@yourcompany.com";

Properties javamailProps = new Properties();
javamailProps.setProperty("mail.host", "!YOUR_MAIL_HOST");
javamailProps.setProperty("mail.user", "!YOUR_MAIL_USER");
javamailProps.setProperty("mail.password", "!YOUR_MAIL_PASSWORD");
javamailProps.setProperty("mail.port","25");

Emailer emailer = new Emailer( engine, javamailProps, from );

emailer.email("person@somedomain.com", "This is a test", "test.vm")
.bind("name", "Chuck")
.bind("password","KutzMutz")
.bind("interestingPlaces","Tahiti")
.send();
System.out.println("Email sent!");
}
}

9/26/2007

Flexjson 1.5 is live!

After a while I've finally released Flexjson 1.5 to the world. I've been running it for quite sometime trying to see how I liked some of the changes. Since I haven't wanted to change any of my decisions I thought that meant it was time to let other people try it out. There were some big additions to the library. The primary was adding wildcard support for including and excluding fields from your objects. This means you can now exclude one or more fields by using the star notation. The biggest example would be to omit the class attribute on JSON objects. For example:


new JSONSerializer().exclude("*.class").serialize( obj );


Pretty simple huh? You can even use wildcards more than once so expressions like:


new JSONSerializer().exclude("foo.*.bar.*").prettyPrint( obj );


Using a plain star ('*') will cause a deep serialization. In previous releases you used the deepSerialize() method to get a deep serialization. You can still use that method. The big thing that changed between releases is the evaluation order of includes and excludes. In prior releases includes were always processed before excludes. So that meant if you did the following:


new JSONSerializer().exclude("*.class").include("my.hobbies").serialize( obj );


In previous release "my.hobbies" would always be processed before *.class, but now they are processed in the order in which you register them. So in 1.5 the order would be to evaluate each field against the "*.class" exclude then evaluate it against "my.hobbies". This enables you do things like:


new JSONSerializer().exclude("foo.phoneNumbers").include("*").prettyPrint( obj );


You might notice a new method I'm using...prettyPrint(). Pretty print is fairly straight forward it formats your JSON output in a pretty to read format. Very nice for debugging.

The other big feature is Transformers. Transformers allow you to register classes that participate in serializing fields. It's best understood as described by a use case. Say we have an object that represents an email, and we want to send it to the browser over JSON. But, emails can have illegal characters for HTML like < or > in fields like the to, from, or cc portions. Before we would have to create a separate method that would HTML encode those characters, and exclude the method that returned those values in plain text. Now we can register a transformer for those fields on the fly. Here's how we can solve this problem:


new JSONSerializer().transform( new HTMLEncoder(), "to", "from", "cc" ).prettyPrint( email );


So the transform() method allows us to register a Transformer called HTMLEncoder. The "to", "from", and "cc" are fields within the email object that we want to run this Transformer on. We can register this Transformer with one or more fields to make it easy on us. The transform() method supports dot notation, just like include and exclude, but doesn't support wildcards.

Transformers can do all sorts of things. Say transform Markdown into HTML from your objects, escape HTML tags and script tags to protect your application, or translating dates into non-numeral format (2007-08-12) or (1/1/2008). Flexjson ships with a HTMLEncoder that you can use out of the box. In the future I hope to add more, especially for security concerns.

This release also has several bug fixes, and performance enhancements. There are some very exciting features in this release. Grab it and see how you like it.

9/06/2007

Javamail parsing made easy

I've recently been working quite heavily with Javamail, and while it's fairly straight forward it does have lots of gotchas with little convenience factors built in. Part of it is just the pain of email (spam, lack of standards compliance, microsoft, outlook, bad servers, etc), but some of it is the lack real power in the API. Quite often people have to find the part of the email the user actually wrote so they can display it to the receiver. Say you're writing Java's Outlook killer, and you want to display HTML when it's available and fall back to plain text if the email didn't send it. To make it a little more complex let's also say we want to put the text portion of our email in Lucene so we can easily search for email just like google. In this case the search text would prefer plain text, but would settle for HTML if it had to. Just the opposite of display.

Here is a rough example of code I found out on the net to do this:


private String findText( Part p ) throws MessagingException, IOException {
if ( p.isMimeType("text/*") ) {
String s = (String)p.getContent();
if( p.isMimeType("text/html") ) {
displayText = s;
if( searchText == null ) {
searchText = displayText.replaceAll("<.*?>","");
}
} else {
searchText = s;
if( displayText == null ) {
displayText = "<pre>" + searchText + "</pre>";
}
}
return s;
} else if (p.isMimeType("multipart/*")) {
Multipart mp = (Multipart)p.getContent();
for (int i = 0; i < mp.getCount(); i++) {
String text = initializeText(mp.getBodyPart(i));
if (text != null)
return text;
}
}
return null;
}


So that sucks. And, yes I wrote it. Well I adapted some code I found off the net, but that only makes it a little less embarrassing. You know what? Don't even look at it. Stop it. It sucks, I know! Why can't you quit staring at it?! It's like a zit on my face. Stop!!!

It sucks because it's trying to do too much. It's trying to do two things in one algorithm. It's trying to find all the parts it's interested in, and it's trying to handle the precedence of HTML overrides plain text. But, that small fact makes it messy because we don't know which we'll encounter first. So, we have to handle both possibilities. It also sucks because sure it finds HTML or text, and it might even work, but it's brittle as all hell. Too much unsafe casting! The first "text/*" mime type will match "text/html" and "text/plain", but it will also match "text/calendar" or "text/xml" which doesn't result in a String when you call getContent(). It won't survive the messy hell that is email. Email is a mess with spammers, bad MTAs, crappy servers that don't reject illegal mail formats, etc. Lots of email uses illegal character sets like "printable-ascii" or "ISO-8859". These charsets aren't included in the JVM, because they aren't real character encodings, but Javamail chokes if it encounters these. So we need to handle the exceptions better.

So that codes sucks, and I know it sucks because I just got done watching it vomit all over a bunch of email I have. Let's simplify. How hard would it be if we just wrote a single algorithm for finding any parts with the given mime types we're interested in. Then we'll make the decision of which one we use. Let's just create a simple method that given a list of mime types it returns all the Part objects that represent those mime types. Something like:


Map mimeTypes = findMimeTypes( part, "text/html", "text/plain" );


That would make it pretty easy. After we have those then we'd just choose HTML over text. Something like the following would be easy to understand:


if( contentTypes.containsKey( "text/plain" ) ) {
try {
Object content = contentTypes.get( "text/plain" ).getContent();
searchText = content.toString();
displayText = "<pre>" + searchText + "</pre>";
} catch( UnsupportedEncodingException ex ) {
logger.warn( ex );
}
}
if( contentTypes.containsKey( "text/html" ) ) {
try {
Object content = contentTypes.get( "text/html" ).getContent();
displayText = content.toString();
if( searchText == null ) {
searchText = displayText.replaceAll("<.*?>","");
}
} catch( UnsupportedEncodingException ex ) {
logger.warn( ex );
}
}
if( searchText == null && displayText == null ) {
searchText = displayText = "Unknown content type. The content of this email was not able to be indexed or read.";
}


That's pretty easy to understand. First we look for plain text. If we find it then set searchText to that value (i.e. text we want to place into Lucene). Next we surround that plain text with <pre> tags for display purposes. Next we look for HTML. If we find HTML then we use that as our displayText. Then if we didn't find any searchText already (i.e. null) then we'll try to strip out all the tags and use that text as our search text.

We also carefully surround each call to getContent() with exception handling in case someone used strange character encoding. Like all those russian emails I keep getting in my Yahoo account.

The last part says if we didn't find any searchable text and we didn't find any displable text, well that means we don't understand what kinda email this is, and we'll just display a message saying so.

That algorithm is pretty easy to follow. Now let's go deeper into the findMimeTypes() function.


public Map findMimeTypes( Part p, String... mimeTypes ) {
Map parts = new HashMap();
findMimeTypesHelper( p, parts, mimeTypes );
return parts;
}

// a little recursive helper function that actually does all the work.
public void findMimeTypesHelper( Part p, Map parts, String... mimeTypes ) {
try {
if (p.isMimeType("multipart/*") ) {
Multipart mp = (Multipart)p.getContent();
for (int i = 0; i < mp.getCount(); i++) {
findContentTypesHelper( mp.getBodyPart(i), parts, mimeTypes );
}
} else {
for( String mimeType : mimeTypes ) {
if( p.isMimeType( mimeType ) && !parts.containsKey( mimeType ) ) {
parts.put( mimeType, p );
}
}
}
} catch( UnsupportedEncodingException ex ) {
logger.warn( p.getContentType(), ex );
}
}

private void findContentTypesHelper( Part p, Map contentTypes, String... mimeTypes ) throws MessagingException, IOException {
try {
if (p.isMimeType("multipart/*") ) {
Multipart mp = (Multipart)p.getContent();
for (int i = 0; mp != null && i < mp.getCount(); i++) {
findContentTypesHelper( mp.getBodyPart(i), contentTypes, mimeTypes );
}
} else {
for( String mimeType : mimeTypes ) {
if( p.isMimeType( mimeType ) ) {
contentTypes.put( mimeType, p );
}
}
}
} catch( UnsupportedEncodingException ex ) {
logger.warn( p.getContentType(), ex );
}
}



Really not a lot of code. Probably easier than the first part huh? All this does is look at the parts mime type and compares it against the multipart mime type. This signifies if we have multiple parts in this email. It then tries to enumerate those emails and recursively calls itself on those parts as well. The next part tries to match that part's mime type against the mime types we're interested in. If we find something we save it off in our map. Notice it only takes the first mime type it encounters of the given mime types. We could look at the Content Disposition and only take Inline HTML and Inline Text, but I've seen problems with the Content Dispositions as well. It's optional so we'd want to generally take the first one we encounter, and that generally works well.

In conclusion I think it's little methods like the findMimeTypes() that make using Javamail so much easier. I wish Sun would add some fun little methods like these when it's building it's specs. Most JCP specs whittle everything down to bare minimum. No frills, no convenience methods, or simplified versions for easy access.

4/22/2007

What makes a DSL a DSL?

I recently released Flexjson to the world, and I've been thrilled by the response I've gotten. But, someone asked why I called Flexjson a Domain Specific Language. It's a good question so I thought I'd write down some ideas around what makes me say something is a DSL vs. when it's just an API.

DSL does get thrown around a lot, and it's hard to always pick out what's a DSL and what's just an API. Hibernate's HQL is definitely a DSL, SQL is a DSL, Phong Shader languages are DSLs, but it gets fuzzier when people say Rails is a DSL or something in LISP. LISP and Ruby are general purpose languages just like Java so how can a library written for a general purpose language be a language too. Isn't that just a well written API? I struggled to answer this question when I first encountered examples of these DSLs.

Martin Fowler wrote some thoughts of this subject and he divided DSLs into two types: external DSL and internal DSL. External DSLs examples are SQL, Hibernate's HQL, ant scripts, or phong shader languages. Internal DSLs would be certain aspects of Rails, or Javascript's Event.Behavior.

External DSLs are glaring examples of DSL. They're easy to spot. Internal DSLs are harder to spot as such. I've decided that Internal DSLs read like a sentence where an API is reads like an algorithm. It's a fuzzy definition, but no less fuzzy than spotting design patterns in code. Said another way DSL should feel more high level, like you're able to produce results in a very simple way.

So what makes an API an API? Well I guess API is not high level. Terrible definition I know, but I think it's pretty accurate. An example of a definite API is JDBC. There's no way to get things done in a simplified way. It's verbose as hell and low level as the day is long. Sorry I get that way about JDBC. ;-) But seriously, it makes me handle all the details. It just gives me access to the database, but I have to manage the connections, the statements, and results sets sure signs I'm dealing with an API.

Flexjson is an internal DSL. It's very simple DSL. There are really only three statements in this language: include, exclude, and serialize. However, while these are just simple methods include and exclude accept strings formatted in dot notation. Dot notation is a simple way of specifying the fields within the object graph. This is probably the part that is most language like. The overall style of Flexjson feels more high level almost language like. So we get something like:


new JSONSerializer().include("bids", "winner.winningBids").serialize(auction);


Let's say I had designed Flexjson like following:


JSONSerializer serializer = new JSONSerializer();
serializer.include( auction.getClass().getField("bids") );
serializer.include( auction.getClass().getField("winner") );
serializer.include( auction.getClass().getField("winner").getDeclaringClass().getField() );
serializer.serialize( auction );


I would say this is starting to look more like an API than a DSL. It doesn't really help me in specifying what I want to happen. It just does the serialization to JSON process. And not as easy to work with which means it's not fun either.

So while Flexjson purpose is to serialize Java objects to JSON. It also tackles the effort of walking the graph for you. Walking the graph is not central to it's purpose, but it's something you have to do as a client. So, Flexjson has a little language that let's you just say in a high-level way what you want to happen. This is what makes your job easier.

The difference between DSL and API sometimes boils down to just style. Java has had a tradition in writing APIs. Some better than others. However, there are a lot of good programmers out there doing things differently, and I think that's a very good thing. I've heard Rubynauts say that you can't create DSLs with Java because it doesn't support all the cool stuff Ruby has. And all that stuff is important to writing DSLs. I happen to disagree with that very much. Java is just as suitable for writing DSLs as any other general purpose language. You just have to work with what you got. I hope to add more on this subject later. DSL vs. API is root in a change in style, but it allows us to do things we couldn't do with our APIs.

It's really about reaching consensus if something is a DSL or not because these things are fuzzy. We're using these APIs to write mini-languages, and exposing our language as APIs so it can be confusing. But, just because it takes the form of an API doesn't mean it's an API. There are lot of benefits to this my favorite is code completion and use of our existing tools. Flexjson I think qualifies as a DSL, albeit a simple one.

4/18/2007

Annoucing: Flexjson for Java 1.5

I recently created a new library for sending Java objects over JSON. It's called Flexjson. Flexjson allows you to control the depth to which you serialize your objects. If you're using libraries like Hibernate for your objects you'll know that many libraries will try to serialize the entire object graph as JSON. This is be a serious performance bottle neck when your object form a connected graph as Hibernate requires. Other libraries force you to manually copy over fields into a seperate object thus creating lots of boiler plate code. Flexjson tries to give you a simple DSL implemented as a normal Java API in order to control with a minimal amount of code what gets translated to JSON.

Check it out: http://flexjson.sourceforge.net